Another way to handle this is not to run commit script at peak time(still pull snapshot periodically). Keeping track of the number of requests, resource utilization, etc.. If the number of request exceeds the threshold, don't commit. Also, how many segments do you see under index dir? High number of segments (implied more files need to be opened) would impact query response time. In that case, you could run optimize script to consolidate into a single segment.
-----Original Message----- From: Lance Norskog [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 12, 2008 11:16 Joe To: solr-user@lucene.apache.org Subject: RE: Query Performance while updating teh index Yes, this is the cache autowarming. We turned this off and staged separate queries that pre-warm our standard queries. We are looking at pulling the query server out of the load balancer during this process; it is the most effective way to give fixed response time. Lance -----Original Message----- From: oleg_gnatovskiy [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 12, 2008 11:07 AM To: solr-user@lucene.apache.org Subject: Re: Query Performance while updating teh index The rsync seems to have nothing to do with slowness, because while the rsync is going on, there isn't any reload occurring, once the files are on the system, it tries a curl request to reload the searcher, which at that point causes the delays. The file transfer probably has nothing to do with this. Does this mean that it happens during warming? Yonik Seeley wrote: > > On Tue, Nov 11, 2008 at 9:31 PM, oleg_gnatovskiy > <[EMAIL PROTECTED]> wrote: >> Hello. We have an index with 15 million documents working on a >> distributed environment, with an index distribution setup. While an >> index on a slave server is being updated, query response times become >> extremely slow (upwards of 5 seconds). Is there any way to decrease >> the hit query response times take while an index is being pushed? > > Can you tell why it's getting slow? Is this during warming, or does > it begin during the actual transfer of the new index? > > One possibility is that the new index being copied forces out parts of > the old index from the OS cache. More memory would help in that > scenario. > > -Yonik > > -- View this message in context: http://www.nabble.com/Query-Performance-while-updating-the-index-tp20452 835p 20467099.html Sent from the Solr - User mailing list archive at Nabble.com.
