We too were getting same issue. We solved it by ensuring that when commit is in progress, no one access the index.
Though SOLR's UpdateRequest does it, we will still read timeout issues because of CommonsHttpSolrServer. If we have another layer which doesnt send the request itself, then you wont get this error. On Tue, Aug 16, 2011 at 5:34 PM, Geir Gullestad Pettersen <gei...@gmail.com>wrote: > Hi, > > We're getting a lot of these timeouts during bulk feeding or a large > document set. We're sending batches of 1000 documents and commiting every > 15 > minutes or for every 10.000 docs, whichever happens first. We find that the > first few commits (after 10'/20'/30' docs) go through without exceptions > (although the "CommonsHttpSolrServer.soTimeout" is set quite high - > 120000ms). The time for a commit to complete seems to increase for every > subsequent commit request, and eventually the timeout value is reached > causing Solrj to throw the java.net.SocketTimeoutException. Increasing the > soTimeout value only delays the problem and it will allow us to send a few > more commits before seeing the exception. > > I've noticed that even though we get these exceptions on commit, documents > seem to be searchable after some time. Does that mean that the commit > process in solr continues even though the socket times out? > > We're also setting "waitFlush=false" and "waitSearcher=false" when > committing, which I would expect to enable a non-blocking commit request to > solr, but on n+1 commits this doesn't aleays happen and this when we're > getting the timeouts. > > I suspect that solr.commit(false,false) is a non-blocking call only if a > all > previous commits are complete. Correct? > > I'm aware that commiting often is not adviced as it is a resouce intensive > process for solr, however it is a requirement that documents are searchable > before the entire document set has been added to solr. Our workaround is to > limit the amount of commits by time and document count "triggers" in our > code. We cannot use the autocommit feature in solrconfig.xml because we > also > need to be able to issue immediate commits for incremental feeding. > > So, my main question is regarding the severity of > "java.net.SocketTimeoutException: Read timed out" on commit. Should I worry > and try to re-issue commit when this happens, or is it safe to ignore it? > > > Thanks for any input! > > Cheers, > Geir >
