Dipti, If I'm reading that correctly, you are optimizing the index on the master before replicating it? There is no need to do that if you are constantly updating your index and replicating it every 10 minutes. Don't optimize, and you'll replicate smaller portion of an index, and thus you won't bust the OS cache on the slave as much. The upgrade to Solr 1.4 and you'll see further benefits from faster searcher warmup times.
Otis -- Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch ----- Original Message ---- > From: dipti khullar <dipti.khul...@gmail.com> > To: solr-user@lucene.apache.org > Sent: Thu, January 21, 2010 11:48:20 AM > Subject: Re: Improvising solr queries > > Hi > > Sorry for getting back late on the thread, but we are focusing on > configuration of master and slave for improving performance issues. > > We have observed following trend on production slaves: > After every 10 minutes the response time increases considerably. In between > all the queries are served by cache. > It seems, after every 10th minute installation and then commit takes time > and hence results in slow response time. > > Following are the logs taken for a complete cycle for master/slave sync up > process: > > 2010/01/21 14:28:02 started by solr > 2010/01/21 14:28:02 command: /opt/solr/solr_master/solr/solr/bin/snapshooter > 2010/01/21 14:28:02 taking snapshot > /opt/solr/solr_master/solr/data/snapshot.20100121142802 > 2010/01/21 14:28:02 ended (elapsed time: 0 sec) > 2010/01/21 14:28:01 started by solr > 2010/01/21 14:28:01 command: /opt/solr/solr_master/solr/solr/bin/optimize > 2010/01/21 14:28:02 ended (elapsed time: 1 sec) > 2010/01/21 14:30:02 started by solr > 2010/01/21 14:30:02 command: /opt/solr/solr_slave/solr/solr/bin/snappuller > 2010/01/21 14:30:06 pulling snapshot snapshot.20100121142802 > 2010/01/21 14:30:14 ended (elapsed time: 12 sec) > 2010/01/21 14:30:14 started by solr > 2010/01/21 14:30:14 command: > /opt/solr/solr_slave/solr/solr/bin/snapinstaller > 2010/01/21 14:30:15 installing snapshot > /opt/solr/solr_slave/solr/data/snapshot.20100121142802 > 2010/01/21 14:30:16 notifing Solr to open a new Searcher > 2010/01/21 14:30:17 ended (elapsed time: 3 sec) > 2010/01/21 14:30:17 started by solr > 2010/01/21 14:30:17 command: /opt/solr/solr_slave/solr/solr/bin/commit > 2010/01/21 14:30:17 ended (elapsed time: 0 sec) > > Response Time at 14:30:24 on: > Slave 1 - 243 > Slave 2 - 111266 > > Are we missing on some configuration. Or perhaps the frequency of execution > of scripts needs to be changed? > Any pointers will be helpful !! > > Thanks > Dipti > > > On Tue, Jan 5, 2010 at 1:16 PM, Shalin Shekhar Mangar < > shalinman...@gmail.com> wrote: > > > On Tue, Jan 5, 2010 at 11:16 AM, dipti khullar > > >wrote: > > > > > > > > This assettype is variable. It can have around 6 values at a time. > > > But this is true that we apply facet mostly on just one field - > > assettype. > > > > > > > > Ian has a good point. You are faceting on assettype and you are also > > filtering on it so you will get only one facet value "Gallery" with a count > > equal to numFound. > > > > > > > Any idea if the use of date range queries is expensive? Also if Shalin > > can > > > put in some comments on > > > "sorting by date was pretty rough on CPU", I can start analyzing sort by > > > date specific queries. > > > > > > > > This is a range search and not a sort. I don't know if range search on > > dates > > is especially costly compared to a range search on any other type. But I do > > know that trie fields in Solr 1.4 are much faster for range searches at the > > cost of more tokens in the index. > > > > With a date field, instead of using NOW, you should always try to round it > > down to the coarsest interval you can use. So if it is possible to use > > NOW/DAY instead of NOW, you should do that. The problem with querying on > > NOW > > is that it is always unique and therefore the query can never be cached > > (actually, it is cached but can never be hit). If you use NOW/DAY, the > > query > > can be cached for a day. > > > > -- > > Regards, > > Shalin Shekhar Mangar. > >
