Re: CPU utilization and query time high on Solr slave when snapshot install

2009-11-03 Thread Mark Miller
Well thats what I get for typing on my iPhone when I'm not sure of my memory - Lance called me out on this - I put "Hmm...I think" because I wasn't positive if I remembered right, but I had thought auto warming just populates based on the old entries (probably because of the javadoc for the CacheRe

Re: CPU utilization and query time high on Solr slave when snapshot install

2009-11-03 Thread Walter Underwood
Message- From: Walter Underwood [mailto:wun...@wunderwood.org] Sent: Monday, November 02, 2009 11:18 PM To: solr-user@lucene.apache.org Subject: Re: CPU utilization and query time high on Solr slave when snapshot install If you are going to pull a new index every 10 minutes, try turning off cache

RE: CPU utilization and query time high on Solr slave when snapshot install

2009-11-03 Thread biku...@sapient.com
workaround impact us as the index size increase? Regds, Bipul -Original Message- From: Walter Underwood [mailto:wun...@wunderwood.org] Sent: Monday, November 02, 2009 11:18 PM To: solr-user@lucene.apache.org Subject: Re: CPU utilization and query time high on Solr slave when snapshot install

Re: CPU utilization and query time high on Solr slave when snapshot install

2009-11-02 Thread Jay Hill
So assuming you set up a few sample sort queries to run in the firstSearcher config, and had very low query volume during that ten minutes so that there were no evictions before a new Searcher was loaded, would those queries run by the firstSearcher be passed along to the cache for the next Searche

Re: CPU utilization and query time high on Solr slave when snapshot install

2009-11-02 Thread Mark Miller
Hmm...I think you have to setup warming queries yourself and that autowarm just copies entries from the old cache to the new cache, rather than issuing queries - the value is how many entries it will copy. Though that's still going to take CPU and time. - Mark http://www.lucidimagination.c

Re: CPU utilization and query time high on Solr slave when snapshot install

2009-11-02 Thread Walter Underwood
If you are going to pull a new index every 10 minutes, try turning off cache autowarming. Your caches are never more than 10 minutes old, so spending a minute warming each new cache is a waste of CPU. Autowarm submits queries to the new Searcher before putting it in service. This will creat

RE: CPU utilization and query time high on Solr slave when snapshot install

2009-11-02 Thread biku...@sapient.com
Hi Solr Gurus, We have solr in 1 master, 2 slave configuration. Snapshot is created post commit, post optimization. We have autocommit after 50 documents or 5 minutes. Snapshot puller runs as a cron every 10 minutes. What we have observed is that whenever snapshot is installed on the slave, we