On 2/17/2014 6:06 PM, Colin Bartolome wrote: > We're using Solr version 4.2.1, in case new functionality has helped > with this issue. > > We have our Solr servers doing automatic soft commits with maxTime=1000. > We also have a scheduled job that triggers a hard commit every fifteen > minutes. When one of these hard commits happens while a soft commit is > already in progress, we get that ubiquitous warning: > > PERFORMANCE WARNING: Overlapping onDeckSearchers=2 > > Recently, we had an occasion to have a second scheduled job also issue a > hard commit every now and then. Since our maxWarmingSearchers value was > set to the default, 2, we occasionally had a hard commit trigger when > two other searchers were already warming up, which led to this: > > org.apache.solr.client.solrj.SolrServerException: No live SolrServers > available to handle this request > > as the servers started responded with a 503 HTTP response. > > It seems like automatic soft commits wait until the hard commits are out > of the way before they proceed. Is there a way to do the same for hard > commits? Since we're passing waitSearcher=true in the update request > that triggers the hard commits, I would expect the request to block > until the server had enough headroom to service the commit. I did not > expect that we'd start getting 503 responses.
Remember this mantra: Hard commits are about durability, soft commits are about visibility. You might already know this, but it is the key to figuring out how to handle commits, whether they are user-triggered or done automatically by the server. With Solr 4.x, it's best to *always* configure autoCommit with openSearcher=false. This does a hard commit but does not open a new searcher. The result: Data is flushed to disk and the current transaction log is closed. New documents will not be searchable after this kind of commit. For maxTime and maxDocs, pick values that won't result in huge transaction logs, which increase Solr startup time. http://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup For document visibility, you can rely on autoSoftCommit, and you indicated that you already have it configured. Decide how long you can wait for new content that has just been indexed. Do you *really* need new data to be searchable within one second? If so, you're good. If not, increase the maxTime value here. Be sure to make the value at least a little bit longer than the amount of time it takes for a soft commit to finish, including cache warmup time. Thanks, Shawn
