On 4/4/2013 12:34 AM, Dotan Cohen wrote:
In the case of maxWarmingSearchers, I would hope that you have your
system set up so that you would never need more than 1 warming searcher
at a time. If you do a commit while a previous commit is still warming,
Solr will try to create a second warming searcher.
How would I set the system up for that? We have very many commits
(every few seconds) and each commit contains a few tens of documents
(mostly smaller than 1 KiB per document). Right now we get about
200-300 searches per minute.
You'll want to ensure that your autowarmCount value on Solr's caches is
low enough that each commit happens quickly. If it takes 5000
milliseconds to warm the caches when you commit, then you want to be
sure that you are committing less often than that, or you'll quickly
reach your maxWarmingSearchers config value. If the commits are
happening VARY quickly, you may need to set autowarmCount to 0, and
possibly disable caches entirely.
I went poking in the code, and it seems that maxWarmingSearchers
defaults to Integer.MAX_VALUE. I'm not sure whether this is a bad
default or not. It does mean that a pathological setup without
maxWarmingSearchers in the config will probably blow up with an
OutOfMemory exception, but is that better or worse than commits that
don't make new documents searchable? I can see arguments either way.
This is interesting, what you found is that the value in the stock
solrconfig.xml file differs from the Solr default value. I think that
this is bad practice: a single default should be decided upon and Solr
should use this value when nothing is specified in solrconfig.xml, and
that _same_value_ should be specified in the stock solrconfig.xml. Is
it not a reasonable assumption that this would be the case?
That was directed more at the other committers. I would argue that
either a low number or a relatively high number should be the default,
but not MAX_VALUE. The example config should have a commented out
section for maxWarmingSearchers that mentions the default. I'm having
the same discussion about maxBooleanClauses on SOLR-4586.
It's possible that this has already been discussed, and that everyone
prefers that a badly configured setup will eventually have a spectacular
blow up with OutOfMemory, rather than semi-silently ignoring commits. A
searcher object contains caches and uses a lot of memory, so having lots
of them around will eventually use up the entire heap.
Thanks,
Shawn
