Looking at the log, that takes one or two seconds after a complete batch reload (master/slave). So that is loading a cold index, all new files. This is not a big index, about a half million book titles.
wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jan 29, 2020, at 1:21 PM, Karl Stoney > <karl.sto...@autotrader.co.uk.INVALID> wrote: > > Out of curiosity, could you define "fast"? > I'm wondering what sort of figures people target their searcher warm time at > ________________________________ > From: Walter Underwood <wun...@wunderwood.org> > Sent: 29 January 2020 21:13 > To: solr-user@lucene.apache.org <solr-user@lucene.apache.org> > Subject: Re: Solr Searcher 100% Latency Spike > > I use a static set of warming queries, about 20 of them. That is fast and > gets a decent amount of the index into file buffers. Your top queries won’t > change much unless you have a news site or a seasonal business. > > Like this: > > <listener event="newSearcher" class="solr.QuerySenderListener"> > <arr name="queries"> > <lst> > <!-- Top non-numeric query words from August 2011 rush --> > <str name="q">introduction</str> > <str name="q">intermediate</str> > <str name="q">fundamentals</str> > <str name="q">understanding</str> > <str name="q">introductory</str> > <str name="q">precalculus</str> > <str name="q">foundations</str> > <str name="q">microeconomics</str> > <str name="q">microbiology</str> > <str name="q">macroeconomics</str> > <str name="q">discovering</str> > <str name="q">international</str> > <str name="q">mathematics</str> > <str name="q">organizational</str> > <str name="q">criminology</str> > <str name="q">developmental</str> > <str name="q">engineering</str> > </lst> > </arr> > </listener> > > wunder > Walter Underwood > wun...@wunderwood.org > https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobserver.wunderwood.org%2F&data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7C48627550665c47efecae08d7a5002b8e%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637159292473223261&sdata=ZCCITDfh2TlR4KKwLzZ%2BVQL1b6%2F3OXewXFS1T3nhlVo%3D&reserved=0 > (my blog) > >> On Jan 29, 2020, at 1:01 PM, Shawn Heisey <apa...@elyograg.org> wrote: >> >> On 1/29/2020 12:44 PM, Karl Stoney wrote: >>> Looking for a bit of support here. When we soft commit (every 10 minutes), >>> we get a latency spike that means response times for solr are loosely >>> double, as you can see in this screenshot: >> >> Attachments almost never make it to the list. We cannot see any of your >> screenshots. >> >>> They do correlate to filterCache warmup, which seem to take between 10s and >>> 30s: >>> We don't have any other caches enabled, due to the high level of >>> cardinality of the queries. >>> The spikes are specifically on /select >>> We have the following autowarm configuration for the filterCache: >>> <filterCache class="solr.FastLRUCache" >>> size="8192" >>> initialSize="8192" >>> cleanupThread="true" >>> autowarmCount="900"/> >> >> Autowarm, especially on filterCache, can be an extremely lengthy process. >> What Solr must do in order to warm the cache here is execute up to 900 >> queries, sequentially, on the new index. That can take a lot of time and >> use a lot of resources like CPU and I/O. >> >> In order to reduce the impact of cache warming, I had to reduce my own >> autowarmCount on the filterCache to 4. >> >> Thanks, >> Shawn > > This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 > Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. > 9439967). This email and any files transmitted with it are confidential and > may be legally privileged, and intended solely for the use of the individual > or entity to whom they are addressed. If you have received this email in > error please notify the sender. This email message has been swept for the > presence of computer viruses.
