On 1/21/2016 8:16 AM, adfel70 wrote: > I am running soft commit on 100 solr docs (the index itself has 3 Billion > docs). > After EXACTLY 10 minutes (for example, start committing on 15:52:55.932, > exception on 16:02:55.976) I am getting several exception of the sort: > org.apache.solr.client.solrj.SolrServerException: Timeout occured while > waiting response from server at: [any instance of solr] ... > Caused by: java.net.SocketTimeoutException: Read timed out > > All the exceptions are from the same instance to other instances in the > cluster. > > I have checked the jetty.xml, the java params and the solr config - and > didn't find any place where a timeout of 10 minutes is configured. > I have tunned the filterCache, queryResultCache and documentCahce to: > size="2048", initialSize="1024", autowarmCount="1024" > I created queries for newSearcher for warm-up > > I have 2 questions: > 1. Can someone point me to the rigth direction for the timeout? > 2. Why could the commit take so long?!
1) The client has been configured with a ten minute socket timeout on TCP connections. This is usually not something that happens by default -- it usually must be configured by somebody. Is this exception happening in your client code, or in the Solr server log? 2) Commits on very large indexes, like what is usually found when the whole index is 3 billion docs, tend to be slow. This could be caused by not having enough memory to effectively cache the index, but more commonly it is caused by large autowarmCount values on large Solr caches, and I believe that is what is happening here. If we assume that your filterCache actually does reach a size of 1024, then the configuration that you have indicated says that every time you commit, Solr will execute the most recent 1024 filter queries in the cache, to warm the new cache. If each filter query takes half a second to execute on a brand new index, then 1024 of them will take eight and a half minutes. Very likely the filters in your cache are taking longer than half a second to execute. Thanks, Shawn
