Hi Eric, thanks for your reply. I am not sure what exactly you mean by warmup queries. But if its related to the settings we are using in solrconfig.xml, following are the configurations for query caching:
<queryResultCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/> Also, as we are using snapinstall script on slaves, which eventually calls commit script. I was just wondering that whether, we need to change the simple commit command to <commit waitFlush="false" waitSearcher="false"/> Otis, we executed a performance test on our local environments for Solr 1.4 but there were not considerable performance improvement. Hence, we have as of now dropped the idea of upgrading to Solr 1.4. Regarding optimization, we initially were not using optimize at all, but then at peak hours load on slaves increased considerably. Hence, we configured the optimize script to get the system running. But we can try this on local environment and then analyze the results. Thanks Dipti On Fri, Jan 22, 2010 at 10:36 AM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > Dipti, > > If I'm reading that correctly, you are optimizing the index on the master > before replicating it? > There is no need to do that if you are constantly updating your index and > replicating it every 10 minutes. > Don't optimize, and you'll replicate smaller portion of an index, and thus > you won't bust the OS cache on the slave as much. > The upgrade to Solr 1.4 and you'll see further benefits from faster > searcher warmup times. > > Otis > -- > Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch > > > > ----- Original Message ---- > > From: dipti khullar <dipti.khul...@gmail.com> > > To: solr-user@lucene.apache.org > > Sent: Thu, January 21, 2010 11:48:20 AM > > Subject: Re: Improvising solr queries > > > > Hi > > > > Sorry for getting back late on the thread, but we are focusing on > > configuration of master and slave for improving performance issues. > > > > We have observed following trend on production slaves: > > After every 10 minutes the response time increases considerably. In > between > > all the queries are served by cache. > > It seems, after every 10th minute installation and then commit takes time > > and hence results in slow response time. > > > > Following are the logs taken for a complete cycle for master/slave sync > up > > process: > > > > 2010/01/21 14:28:02 started by solr > > 2010/01/21 14:28:02 command: > /opt/solr/solr_master/solr/solr/bin/snapshooter > > 2010/01/21 14:28:02 taking snapshot > > /opt/solr/solr_master/solr/data/snapshot.20100121142802 > > 2010/01/21 14:28:02 ended (elapsed time: 0 sec) > > 2010/01/21 14:28:01 started by solr > > 2010/01/21 14:28:01 command: /opt/solr/solr_master/solr/solr/bin/optimize > > 2010/01/21 14:28:02 ended (elapsed time: 1 sec) > > 2010/01/21 14:30:02 started by solr > > 2010/01/21 14:30:02 command: > /opt/solr/solr_slave/solr/solr/bin/snappuller > > 2010/01/21 14:30:06 pulling snapshot snapshot.20100121142802 > > 2010/01/21 14:30:14 ended (elapsed time: 12 sec) > > 2010/01/21 14:30:14 started by solr > > 2010/01/21 14:30:14 command: > > /opt/solr/solr_slave/solr/solr/bin/snapinstaller > > 2010/01/21 14:30:15 installing snapshot > > /opt/solr/solr_slave/solr/data/snapshot.20100121142802 > > 2010/01/21 14:30:16 notifing Solr to open a new Searcher > > 2010/01/21 14:30:17 ended (elapsed time: 3 sec) > > 2010/01/21 14:30:17 started by solr > > 2010/01/21 14:30:17 command: /opt/solr/solr_slave/solr/solr/bin/commit > > 2010/01/21 14:30:17 ended (elapsed time: 0 sec) > > > > Response Time at 14:30:24 on: > > Slave 1 - 243 > > Slave 2 - 111266 > > > > Are we missing on some configuration. Or perhaps the frequency of > execution > > of scripts needs to be changed? > > Any pointers will be helpful !! > > > > Thanks > > Dipti > > > > > > On Tue, Jan 5, 2010 at 1:16 PM, Shalin Shekhar Mangar < > > shalinman...@gmail.com> wrote: > > > > > On Tue, Jan 5, 2010 at 11:16 AM, dipti khullar > > > >wrote: > > > > > > > > > > > This assettype is variable. It can have around 6 values at a time. > > > > But this is true that we apply facet mostly on just one field - > > > assettype. > > > > > > > > > > > Ian has a good point. You are faceting on assettype and you are also > > > filtering on it so you will get only one facet value "Gallery" with a > count > > > equal to numFound. > > > > > > > > > > Any idea if the use of date range queries is expensive? Also if > Shalin > > > can > > > > put in some comments on > > > > "sorting by date was pretty rough on CPU", I can start analyzing sort > by > > > > date specific queries. > > > > > > > > > > > This is a range search and not a sort. I don't know if range search on > > > dates > > > is especially costly compared to a range search on any other type. But > I do > > > know that trie fields in Solr 1.4 are much faster for range searches at > the > > > cost of more tokens in the index. > > > > > > With a date field, instead of using NOW, you should always try to round > it > > > down to the coarsest interval you can use. So if it is possible to use > > > NOW/DAY instead of NOW, you should do that. The problem with querying > on > > > NOW > > > is that it is always unique and therefore the query can never be cached > > > (actually, it is cached but can never be hit). If you use NOW/DAY, the > > > query > > > can be cached for a day. > > > > > > -- > > > Regards, > > > Shalin Shekhar Mangar. > > > > >
