On 6/27/2014 12:00 PM, IJ wrote: > These observations leads me to believe the following (still a theory): > 1. There is something thats breaking / disrupting inter-node communication > between NodeX and NodeY > Could this be a firewall or something similar ? > > 2. Whenever NodeX remains idle for more than 1.5 minutes - its connection to > NodeY is dropped (I can't see anything in the logs to that effect though), > and when the next request comes in - it takes 5 seconds to recreate the > connection > This 1.5 minute window and the 5 second delay are pretty consistent
The 1.5 minutes may be something like HTTP Keepalive. I don't know for sure that Solr uses keepalive for inter-node cloud traffic, but it would be a smart thing to do. The five second problem after a connection drop sounds like a DNS lookup issue, and I found this: http://serverfault.com/questions/339791/5-second-resolving-delay It was the first result on a google search for "dns timeout 5 seconds". Thanks, Shawn