On Sat, Jan 8, 2011 at 3:44 AM, Burton-West, Tom <tburt...@umich.edu> wrote: > Hello all, > > We are getting intermittent socket timeout errors (see below). Out of about > 600,000 indexing requests, 30 returned these socket timeout errors. We > haven't been able to correlate these with large merges, which tends to slow > down the indexing response rate. > > Does anyone know where we might look to determine the cause? [...]
We also experienced such timeouts when our indexing was taking some 7-8 hours. It almost certainly had to do with either the network, or the database server, but we were also unable to track it down. You could try monitoring the network for glitches, and monitoring load on the database server to see if there is a correlation with excessive load. What database are you using? If it is Microsoft SQL server, we found that moving to the jtds JDBC driver, rather than the Microsoft one helped reduce such errors, though it did not eliminate them. What we finally ended up doing is sharding the indexing task, both at the database end, and at the Solr end. Then, we just detect any indexing errors, and reindex that shard, which is much faster than a complete reindexing. Along with the above improvements, the problem became quite minor. Regards, Gora
