Hi, We're getting a lot of these timeouts during bulk feeding or a large document set. We're sending batches of 1000 documents and commiting every 15 minutes or for every 10.000 docs, whichever happens first. We find that the first few commits (after 10'/20'/30' docs) go through without exceptions (although the "CommonsHttpSolrServer.soTimeout" is set quite high - 120000ms). The time for a commit to complete seems to increase for every subsequent commit request, and eventually the timeout value is reached causing Solrj to throw the java.net.SocketTimeoutException. Increasing the soTimeout value only delays the problem and it will allow us to send a few more commits before seeing the exception.
I've noticed that even though we get these exceptions on commit, documents seem to be searchable after some time. Does that mean that the commit process in solr continues even though the socket times out? We're also setting "waitFlush=false" and "waitSearcher=false" when committing, which I would expect to enable a non-blocking commit request to solr, but on n+1 commits this doesn't aleays happen and this when we're getting the timeouts. I suspect that solr.commit(false,false) is a non-blocking call only if a all previous commits are complete. Correct? I'm aware that commiting often is not adviced as it is a resouce intensive process for solr, however it is a requirement that documents are searchable before the entire document set has been added to solr. Our workaround is to limit the amount of commits by time and document count "triggers" in our code. We cannot use the autocommit feature in solrconfig.xml because we also need to be able to issue immediate commits for incremental feeding. So, my main question is regarding the severity of "java.net.SocketTimeoutException: Read timed out" on commit. Should I worry and try to re-issue commit when this happens, or is it safe to ignore it? Thanks for any input! Cheers, Geir
