Re: Solr cache considerations

2013-01-23 Thread Otis Gospodnetic
I think the attachment got stripped. Here it is: http://www.flickr.com/photos/otis/8409088080/in/photostream Otis -- Solr & ElasticSearch Support http://sematext.com/ On Tue, Jan 22, 2013 at 12:36 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Same here - I've seen some documen

Re: Solr cache considerations

2013-01-22 Thread Otis Gospodnetic
Same here - I've seen some document caches that were huge and highly utilized. Check out the screenshot of the SPM for Solr dashboard that shows pretty high hit rates on all caches. I've circled the parts to look at. ML manager may strip the attachment, of course. :) In addition to multiple in-

Re: Solr cache considerations

2013-01-21 Thread Erick Erickson
Hmm, interesting. I'll have to look closer... On Sun, Jan 20, 2013 at 3:50 PM, Walter Underwood wrote: > I routinely see hit rates over 75% on the document cache. Perhaps yours is > too small. Mine is set at 10240 entries. > > wunder > > On Jan 20, 2013, at 8:08 AM, Erick Erickson wrote: > >> Ab

Re: Solr cache considerations

2013-01-20 Thread Walter Underwood
I routinely see hit rates over 75% on the document cache. Perhaps yours is too small. Mine is set at 10240 entries. wunder On Jan 20, 2013, at 8:08 AM, Erick Erickson wrote: > About your question about document cache: Typically the document cache > has a pretty low hit-ratio. I've rarely, if ev

Re: Solr cache considerations

2013-01-20 Thread Isaac Hebsh
Wow Erick, The MMap acrtivle is a very fundamental one. Totaly changed my view. It must be mentioned in SolrPerformanceFactors in SolrWiki... I'm sorry I did not know it before. Thank you a lot. I promise to share my results then my cart will start to fly :) On Sun, Jan 20, 2013 at 6:08 PM, Erick

Re: Solr cache considerations

2013-01-20 Thread Erick Erickson
About your question about document cache: Typically the document cache has a pretty low hit-ratio. I've rarely, if ever, seen it get hit very often. And remember that this cache is only hit when assembling the response for a few documents (your page size). Bottom line: I wouldn't worry about this

Re: Solr cache considerations

2013-01-19 Thread Isaac Hebsh
Ok. Thank you everyone for your helpful answers. I understand that fieldValueCache is not used for resolving queries. Is there any cache that can help this basic scenario (a lot of different queries, on a small set of fields)? Does Lucene's FieldCache help (implicitly)? How can I use RAM to reduce

Re: Solr cache considerations

2013-01-18 Thread Tomás Fernández Löbbe
No, the fieldValueCache is not used for resolving queries. Only for multi-token faceting and apparently for the stats component too. The document cache maintains in memory the stored content of the fields you are retrieving or highlighting on. It'll hit if the same document matches the query multip

Re: Solr cache considerations

2013-01-17 Thread Isaac Hebsh
Unfortunately, it seems ( http://lucene.472066.n3.nabble.com/Nrt-and-caching-td3993612.html) that these caches are not per-segment. In this case, I want to (soft) commit less frequently. Am I right? Tomás, as the fieldValueCache is very similar to lucene's FieldCache, I guess it has a big contribu

Re: Solr cache considerations

2013-01-17 Thread Tomás Fernández Löbbe
I think fieldValueCache is not per segment, only fieldCache is. However, unless I'm missing something, this cache is only used for faceting on multivalued fields On Thu, Jan 17, 2013 at 8:58 PM, Erick Erickson wrote: > filterCache: This is bounded by 1M * (maxDoc) / 8 * (num filters in > cache).

Re: Solr cache considerations

2013-01-17 Thread Erick Erickson
filterCache: This is bounded by 1M * (maxDoc) / 8 * (num filters in cache). Notice the /8. This reflects the fact that the filters are represented by a bitset on the _internal_ Lucene ID. UniqueId has no bearing here whatsoever. This is, in a nutshell, why warming is required, the internal Lucene I