Hmmm, rather than hit the ping query, why not just send in a real query and only let the queued ones through after the response?
Just a random thought Erick On Sat, Nov 10, 2012 at 2:53 PM, Amit Nithian <anith...@gmail.com> wrote: > Yes but the problem is that if user facing queries are hitting a server > that is warming up and isn't being serviced quickly, then you could > potentially bring down your site if all the front end threads are blocked > on Solr queries b/c those queries are waiting (presumably at the container > level since the filter hasn't finished its init() sequence) for the warming > to complete (this is especially notorious when your front end is rails). > This is why your ping to enable/disable a server from the load balancer has > to be accurate with regards to whether or not a server is truly ready and > warm. > > I think what I am gathering from this discussion is that the server is > warming up, the ping is going through and tells the load balancer this > server is ready, user queries are hitting this server and are queued > waiting for the firstSearcher to finish (say these initial user queries are > to respond in 500-1000ms) that's terrible for performance. > > Alternatively, if you have a bunch of servers behind a load balancer, you > want this one server (or block of servers depending on your deployment) to > be reasonably sure that user queries will return in a decent time (whatever > you define decent to be) hence why this matters. > > Let me know if I am missing anything. > > Thanks > Amit > > > On Sat, Nov 10, 2012 at 10:03 AM, Erick Erickson <erickerick...@gmail.com > >wrote: > > > Why does it matter? The whole idea of firstSearcher queries is to warm up > > your system as fast as possible. The theory is that upon restarting the > > server, let's bet this stuff going immediately... They were never > intended > > (as far as I know) to complete before any queries were handled. As an > > aside, I'm not quite sure I understand why pings during the warmup are a > > problem. > > > > But anyway. firstSearcher is particularly relevant because the > > autowarmCount settings on your caches are irrelevant when starting the > > server, there's no history to autowarm.... > > > > But, there's no good reason _not_ to let queries through while > > firstSearcher is doing it's tricks, they just get into the queue and are > > served as quickly as they may. That might be some time since, as you say, > > they may not get serviced until the expensive parts get filled. But I > don't > > think having them be serviced is doing any harm. > > > > Now, newSearcher and autowarming of the caches is a completely different > > beast since having the old searchers continue serving requests until the > > warmups _does_ directly impact the user, they don't see random slowness > > because a searcher is being opened. > > > > So I guess my real question is whether you're seeing a measurable problem > > or if this is a red herring.... > > > > FWIW, > > Erick > > > > > > On Thu, Nov 8, 2012 at 2:54 PM, Aaron Daubman <daub...@gmail.com> wrote: > > > > > Greetings, > > > > > > I have several custom QueryComponents that have high one-time startup > > costs > > > (hashing things in the index, caching things from a RDBMS, etc...) > > > > > > Is there a way to prevent solr from accepting connections before all > > > QueryComponents are "ready"? > > > > > > Especially, since many of our instance are load-balanced (and > > > added-in/removed automatically based on admin/ping responses) > preventing > > > ping from answering prior to all custom QueryComponents being ready > would > > > be ideal... > > > > > > Thanks, > > > Aaron > > > > > >
