On 4/23/2013 3:44 PM, gustavonasu wrote:
If I understand well the autoWarmCount is the number of elements used from the cache for new searches. I guess that this isn't the problem because after the commit property increases on the "UPDATE HANDLERS" (admin UI) I can see the new docs in the searches result.
Autowarming is the process of using entries in the cache on the old searcher to warm up the caches on new searcher. If one of the most recently used entries in your old filterCache (before commit) is "inStock:true" then a search for that filter will be executed on the new searcher and the results stored in the new filterCache. It has nothing to do with being able to see recently added or deleted documents in searches.
Unfortunately I can't increase the java heap on the servers right now. So I was think to change some configurations to release some memory. For example, we could decrease the maxBufferedDocs value. Do you know if it will be effective?
One of the biggest RAM consumers in Solr is Lucene's FieldCache, but you can't configure that. If that's becoming a problem, you have to change the nature of your queries so they require less memory.
There are two main configuration points you can use to control Solr RAM usage - ramBufferSizeMB and the various Solr caches. The maxBufferedDocs option is deprecated, replaced by ramBufferSizeMB. I would just remove it. Take a look at a recent Solr version like 4.2.1, look at the example solrconfig.xml for collection1.
The default ramBufferSizeMB in newer Solr versions is 100. Unless you are indexing incredibly large documents, this is plenty, and you probably won't make things go any faster by increasing it. Decreasing it is one way of reducing Solr's RAM usage. The previous default was 32.
For Solr's caches, you normally don't need very big numbers. If you have tons of RAM to spare, making them large might provide some benefit. What I am saying below is only general advice, there are sometimes very good reasons for making things much larger. If you do choose the change the numbers, be sure to keep an eye on your query times to make sure they are still acceptable. It all depends on what kind of queries your users send.
The documentCache offers questionable benefits. It may be better to let the operating system cache your stored fields. You probably don't need more than a thousand or two entries in the documentCache, and I've seen some people saying that a value of zero might be preferable. I'm not sure I would completely disable it without a LOT of testing.
The queryResultCache can really help Solr perform well, but if it's too big, then it will just eat RAM without providing any actual benefit. Having more than a couple thousand entries here is probably unnecessary. Mine is set to 512. Also look for the queryResultWindowSize parameter.
The filterCache is amazing when it comes to performance enhancement, but also a major source of autowarming headaches. I have this set to a max size of 64, with an autowarmCount of 4. I've got very complex filters, and even warming only four entries, it still sometimes takes 20-30 seconds. The filterCache does get used for some things besides filter queries, but normally you don't need very many entries here.
Thanks, Shawn
