What this looks like (and I've only glanced) is that your index updates are causing a new searcher to be opened, and the first few queries after the reopen will be slow.
Have you tried warmup queries after the reopen? FWIW Erick On Thu, Jan 21, 2010 at 11:48 AM, dipti khullar <dipti.khul...@gmail.com>wrote: > Hi > > Sorry for getting back late on the thread, but we are focusing on > configuration of master and slave for improving performance issues. > > We have observed following trend on production slaves: > After every 10 minutes the response time increases considerably. In between > all the queries are served by cache. > It seems, after every 10th minute installation and then commit takes time > and hence results in slow response time. > > Following are the logs taken for a complete cycle for master/slave sync up > process: > > 2010/01/21 14:28:02 started by solr > 2010/01/21 14:28:02 command: > /opt/solr/solr_master/solr/solr/bin/snapshooter > 2010/01/21 14:28:02 taking snapshot > /opt/solr/solr_master/solr/data/snapshot.20100121142802 > 2010/01/21 14:28:02 ended (elapsed time: 0 sec) > 2010/01/21 14:28:01 started by solr > 2010/01/21 14:28:01 command: /opt/solr/solr_master/solr/solr/bin/optimize > 2010/01/21 14:28:02 ended (elapsed time: 1 sec) > 2010/01/21 14:30:02 started by solr > 2010/01/21 14:30:02 command: /opt/solr/solr_slave/solr/solr/bin/snappuller > 2010/01/21 14:30:06 pulling snapshot snapshot.20100121142802 > 2010/01/21 14:30:14 ended (elapsed time: 12 sec) > 2010/01/21 14:30:14 started by solr > 2010/01/21 14:30:14 command: > /opt/solr/solr_slave/solr/solr/bin/snapinstaller > 2010/01/21 14:30:15 installing snapshot > /opt/solr/solr_slave/solr/data/snapshot.20100121142802 > 2010/01/21 14:30:16 notifing Solr to open a new Searcher > 2010/01/21 14:30:17 ended (elapsed time: 3 sec) > 2010/01/21 14:30:17 started by solr > 2010/01/21 14:30:17 command: /opt/solr/solr_slave/solr/solr/bin/commit > 2010/01/21 14:30:17 ended (elapsed time: 0 sec) > > Response Time at 14:30:24 on: > Slave 1 - 243 > Slave 2 - 111266 > > Are we missing on some configuration. Or perhaps the frequency of execution > of scripts needs to be changed? > Any pointers will be helpful !! > > Thanks > Dipti > > > On Tue, Jan 5, 2010 at 1:16 PM, Shalin Shekhar Mangar < > shalinman...@gmail.com> wrote: > > > On Tue, Jan 5, 2010 at 11:16 AM, dipti khullar <dipti.khul...@gmail.com > > >wrote: > > > > > > > > This assettype is variable. It can have around 6 values at a time. > > > But this is true that we apply facet mostly on just one field - > > assettype. > > > > > > > > Ian has a good point. You are faceting on assettype and you are also > > filtering on it so you will get only one facet value "Gallery" with a > count > > equal to numFound. > > > > > > > Any idea if the use of date range queries is expensive? Also if Shalin > > can > > > put in some comments on > > > "sorting by date was pretty rough on CPU", I can start analyzing sort > by > > > date specific queries. > > > > > > > > This is a range search and not a sort. I don't know if range search on > > dates > > is especially costly compared to a range search on any other type. But I > do > > know that trie fields in Solr 1.4 are much faster for range searches at > the > > cost of more tokens in the index. > > > > With a date field, instead of using NOW, you should always try to round > it > > down to the coarsest interval you can use. So if it is possible to use > > NOW/DAY instead of NOW, you should do that. The problem with querying on > > NOW > > is that it is always unique and therefore the query can never be cached > > (actually, it is cached but can never be hit). If you use NOW/DAY, the > > query > > can be cached for a day. > > > > -- > > Regards, > > Shalin Shekhar Mangar. > > >
