Re: Solr 7.3 cluster issue

2019-12-28 Thread David Barnett
Hi Jan et all clusterstate shows all cores and replicas on node 1 and 2 but node 0 is empty. All three nodes live_nodes shows the correct 3 node addresses. Thanks for the advice, we will use a 4th node. On 28 Dec 2019, 14:10 +, Jan Høydahl , wrote: > Wonder what clusterstate actually says. I

Re: Solr 7.3 cluster issue

2019-12-28 Thread Erick Erickson
+1 to Jan’s comments, especially the idea of adding a 4th node and doing your ADDREPLICAs to that before doing the DELETEREPLICAS for the replicas on the sick node. I’ve used this to bring clusters back to health. This assumes you have at least one active leader for all shards. That ZK error is

Re: Solr 7.3 cluster issue

2019-12-28 Thread Jan Høydahl
Wonder what clusterstate actually says. I can think of two things that could possibly heal the cluster: A rolling restart of all nodes may make Solr heal itself, but the risk is that some shards may not have a replica and if you get stuck in recovery during restart you have downtime. Another

Solr 7.3 cluster issue

2019-12-27 Thread David Barnett
Happy holidays folks, we have a production deployment usage Solr 7.3 in a three node cluster we have a number of collections setup, three shards with a replica factor of 2. The system has been fine, but we experienced issues with disk space one of the nodes. Node 0 starts but does not show any