Re: Solr node not found in ZK live_nodes

2016-12-08 Thread Susheel Kumar
This happens some time that one of the node goes down but then it gets registered as Leader/Active. Does the Cloud View shows anything about this node (Recovering/Down/Recovery Failed etc.) and are you able to perform query to just this shard/node directly? Susheel On Wed, Dec 7, 2016 at 10:13 P

Re: Solr node not found in ZK live_nodes

2016-12-07 Thread Mark Miller
That already happens. The ZK client itself will reconnect when it can and trigger everything to be setup like when the cluster first starts up, including a live node and leader election, etc. You may have hit a bug or something else missing from this conversation, but reconnecting after losing the

Re: Solr node not found in ZK live_nodes

2016-12-06 Thread Manohar Sripada
Thanks Erick! Should I create a JIRA issue for the same? Regarding the logs, I have changed the log level to WARN. That may be the reason, I couldn't get anything from it. Thanks, Manohar On Tue, Dec 6, 2016 at 9:58 PM, Erick Erickson wrote: > Most likely reason is that the Solr node in questi

Re: Solr node not found in ZK live_nodes

2016-12-06 Thread Erick Erickson
Most likely reason is that the Solr node in question, was not reachable thus it was removed from live_nodes. Perhaps due to temporary network glitch, long GC pause or the like. If you're rolling your logs over it's quite possible that any illuminating messages were lost. The default 4M size for eac

Solr node not found in ZK live_nodes

2016-12-06 Thread Manohar Sripada
We have a 16 node cluster of Solr (5.2.1) and 5 node Zookeeper (3.4.6). All the Solr nodes were registered to Zookeeper (ls /live_nodes) when setup was done 3 months back. Suddenly, few days back our search started failing because one of the solr node(consider s16) was not seen in Zookeeper, i.e.,