Out of curiosity, could you define "fast"?
I'm wondering what sort of figures people target their searcher warm time at
________________________________
From: Walter Underwood <wun...@wunderwood.org>
Sent: 29 January 2020 21:13
To: solr-user@lucene.apache.org <solr-user@lucene.apache.org>
Subject: Re: Solr Searcher 100% Latency Spike
I use a static set of warming queries, about 20 of them. That is fast and gets
a decent amount of the index into file buffers. Your top queries won’t change
much unless you have a news site or a seasonal business.
Like this:
<listener event="newSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
<!-- Top non-numeric query words from August 2011 rush -->
<str name="q">introduction</str>
<str name="q">intermediate</str>
<str name="q">fundamentals</str>
<str name="q">understanding</str>
<str name="q">introductory</str>
<str name="q">precalculus</str>
<str name="q">foundations</str>
<str name="q">microeconomics</str>
<str name="q">microbiology</str>
<str name="q">macroeconomics</str>
<str name="q">discovering</str>
<str name="q">international</str>
<str name="q">mathematics</str>
<str name="q">organizational</str>
<str name="q">criminology</str>
<str name="q">developmental</str>
<str name="q">engineering</str>
</lst>
</arr>
</listener>
wunder
Walter Underwood
wun...@wunderwood.org
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobserver.wunderwood.org%2F&data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7C48627550665c47efecae08d7a5002b8e%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637159292473223261&sdata=ZCCITDfh2TlR4KKwLzZ%2BVQL1b6%2F3OXewXFS1T3nhlVo%3D&reserved=0
(my blog)
> On Jan 29, 2020, at 1:01 PM, Shawn Heisey <apa...@elyograg.org> wrote:
>
> On 1/29/2020 12:44 PM, Karl Stoney wrote:
>> Looking for a bit of support here. When we soft commit (every 10 minutes),
>> we get a latency spike that means response times for solr are loosely
>> double, as you can see in this screenshot:
>
> Attachments almost never make it to the list. We cannot see any of your
> screenshots.
>
>> They do correlate to filterCache warmup, which seem to take between 10s and
>> 30s:
>> We don't have any other caches enabled, due to the high level of cardinality
>> of the queries.
>> The spikes are specifically on /select
>> We have the following autowarm configuration for the filterCache:
>> <filterCache class="solr.FastLRUCache"
>> size="8192"
>> initialSize="8192"
>> cleanupThread="true"
>> autowarmCount="900"/>
>
> Autowarm, especially on filterCache, can be an extremely lengthy process.
> What Solr must do in order to warm the cache here is execute up to 900
> queries, sequentially, on the new index. That can take a lot of time and use
> a lot of resources like CPU and I/O.
>
> In order to reduce the impact of cache warming, I had to reduce my own
> autowarmCount on the filterCache to 4.
>
> Thanks,
> Shawn
This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1
Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No.
9439967). This email and any files transmitted with it are confidential and may
be legally privileged, and intended solely for the use of the individual or
entity to whom they are addressed. If you have received this email in error
please notify the sender. This email message has been swept for the presence of
computer viruses.
