Hi Toke, Thanks for the explanation. I will prefer the memory-based limit too. At first I got confused with that too, thinking that the setting of 2000 means 2GB.
Regards, Edwin On 17 August 2016 at 17:40, Toke Eskildsen <t...@statsbiblioteket.dk> wrote: > On Wed, 2016-08-17 at 11:02 +0800, Zheng Lin Edwin Yeo wrote: > > Would like to check, do I need to increase my Java Heap size for > > Solr, if I plan to increase my filterCache size in solrconfig.xml? > > > > I'm using Solr 6.1.0 > > It _seems_ that you can specify a limit in megabytes when using > LRUCache in Solr 5.2+: https://issues.apache.org/jira/browse/SOLR-7372 > > The documentation only mentions it for queryResultCache, but I do not > know if that is intentional (i.e. it does not work for filterCache) or > a shortcoming of the documentation: > https://cwiki.apache.org/confluence/display/solr/Query+Settings+in+Solr > Config > > If it does work for filterCache too (using LRUCache, I guess), then > that would be a much better way of limiting cache size than the highly > insufficient count-based limiter. > > > I say "highly insufficient" because filter cache entries are not of > equal size. With small sets they are stored as sparse, using a > relatively small amount of memory. For larger sets they are stored as > bitmaps, taking up ~1K + maxdoc/8 bytes as Erick describes. > > So a fixed upper limit measured in counts needs to be adjusted to worst > case, meaning maxdoc/8, to ensure stability. In reality most of the > filter cache entries are small, meaning that there is plenty of heap > not being used. This leads people to over-allocate the max size for the > filterCache (very understandable) , resulting in setups that are only > stable as long as there are not too many large filter sets stores. > Leaving it to chance really. > > I would prefer the count-based limit to be deprecated for the > filterCache, or at least warned against, in favour of memory-based. > > - Toke Eskildsen, State and University Library, Denmark > >
