Hey Erick. Our CI process will do the following whenever it detects either a schema or solrconfig change:
* Upload the new configuration to zookeeper * Link it to the existing collection * Reload collection This might be for something as simply as tweaking the caches. I'd say on average, this happens a couple of times per week - however it always causes that spike. As we're a 24/7 business I am trying to mitigate it as much as possible. We've invested quite a bit of time in newSearcher queries but aren't really getting what we need from it. Does reload collection bin of the current, warm searcher before the newSearcher is ready? Therefore if useColdSearcher=false requests are blocked until the collection warming is complete (rather than still serving from the old searcher). Karl ________________________________ From: Erick Erickson <erickerick...@gmail.com> Sent: 03 March 2020 15:28 To: solr-user <solr-user@lucene.apache.org> Subject: Re: collect reload causes a big latency spike Reload throws most everything away, there's no provision for autowarming from the old caches. Consider that schema and solrconfig may have changed, so there would be lots of places could go wrong. I have to ask why you're reloading often enough to see this? This is a heavyweight action, really intended to be used very rarely. If it's because you're using schemaless mode I recommend you don't do that. If you insist on frequently reloading, though, you can configure the firstsearcher with static warming queries, that event is intended exactly to autowarm cold searchers. But that just means the replica won't even start serving queries until autowarming is complete, so if you're reloading often enough that your users are noticing spikes, I don't think that will help. So my question returns: why are you reloading so often? Sounds like an XY problem... Best, Erick On Tue, Mar 3, 2020, 07:38 Karl Stoney <karl.sto...@autotrader.co.uk.invalid> wrote: > Hi Everyone, > When we use the solr collections API to reload a collection, we get a > large latency spike in requests. I'm surprised by this because when we do > new soft commits, our warming means they're near enough undetectable. > > Could anyone confirm if solr collection reload not use the filterCache and > queryResultCache auto warm, and instead give you a "newSearcher"? As > that's how it feels. > > And if that's the case; surely it should do? > > Thanks > Karl > This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: > 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England > No. 9439967). This email and any files transmitted with it are confidential > and may be legally privileged, and intended solely for the use of the > individual or entity to whom they are addressed. If you have received this > email in error please notify the sender. This email message has been swept > for the presence of computer viruses. > This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 9439967). This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
