The first question is _why_ is your disk full? Older versions of Solr could, for instance, accumulate solr console log files forever. If that's the case, just stop the Solr instance on that node, remove the Solr log files, fix the log4j.properties file to not append to console forever and you're done.
If it's more complicated, I'd 1> Spin up a new Solr node with adequate disk space. 2> use the Collections API ADDREPLICA command to add new replicas to this node. The indexes will be synchronized etc. 3> once <2> is complete and the newly added replicas are green, issue a DELETEREPLICA on the replicas on the Solr node in distress. You could issue the DELETEREPLICA before creating a new Solr instance on a new node, but doing it in the order I indicated has the fewest chances for doing something awful like deleting the replica on the good node by mistake without having a healthy replica in your cluster.... Best, Erick On Mon, Dec 19, 2016 at 5:56 PM, Brian Narsi <bnars...@gmail.com> wrote: > We have had a situation where Solr node was in distress due to hard drive > being full and the queries became very slow. Since our Solr cluster has two > nodes with indexes being fully available on both the nodes, we think that > one good solution would be to just stop the Solr instance on a distressed > node. That way the other node will continue to serve queries until the > distressed node is fixed. > > Can folks please share their experiences in such a situation? > > What are some good ways to handle this? How/when should we detect that the > host (linux) is in distress and then stop the solr node? > > Thanks,