I use a static set of warming queries, about 20 of them. That is fast and gets
a decent amount of the index into file buffers. Your top queries won’t change
much unless you have a news site or a seasonal business.
Like this:
<listener event="newSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
<!-- Top non-numeric query words from August 2011 rush -->
<str name="q">introduction</str>
<str name="q">intermediate</str>
<str name="q">fundamentals</str>
<str name="q">understanding</str>
<str name="q">introductory</str>
<str name="q">precalculus</str>
<str name="q">foundations</str>
<str name="q">microeconomics</str>
<str name="q">microbiology</str>
<str name="q">macroeconomics</str>
<str name="q">discovering</str>
<str name="q">international</str>
<str name="q">mathematics</str>
<str name="q">organizational</str>
<str name="q">criminology</str>
<str name="q">developmental</str>
<str name="q">engineering</str>
</lst>
</arr>
</listener>
wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Jan 29, 2020, at 1:01 PM, Shawn Heisey <apa...@elyograg.org> wrote:
>
> On 1/29/2020 12:44 PM, Karl Stoney wrote:
>> Looking for a bit of support here. When we soft commit (every 10 minutes),
>> we get a latency spike that means response times for solr are loosely
>> double, as you can see in this screenshot:
>
> Attachments almost never make it to the list. We cannot see any of your
> screenshots.
>
>> They do correlate to filterCache warmup, which seem to take between 10s and
>> 30s:
>> We don't have any other caches enabled, due to the high level of cardinality
>> of the queries.
>> The spikes are specifically on /select
>> We have the following autowarm configuration for the filterCache:
>> <filterCache class="solr.FastLRUCache"
>> size="8192"
>> initialSize="8192"
>> cleanupThread="true"
>> autowarmCount="900"/>
>
> Autowarm, especially on filterCache, can be an extremely lengthy process.
> What Solr must do in order to warm the cache here is execute up to 900
> queries, sequentially, on the new index. That can take a lot of time and use
> a lot of resources like CPU and I/O.
>
> In order to reduce the impact of cache warming, I had to reduce my own
> autowarmCount on the filterCache to 4.
>
> Thanks,
> Shawn
