A colleague of mine was testing how solrcloud replica recovery works. We have had a lot of issues with replicas going into recovery mode, replicas down and in recovery failed states. So to test, he deleted a healthy replica in one of our development. First the delete operation timed out, but the replica appears to be gone. However, addReplica always fails with this error:
Error CREATEing SolrCore 'sial-content-citations_shard1_replica1': Unable to create core [sial-content-citations_shard1_replica1] Caused by: Lock held by this virtual machine: /var/solr/data/sial-content- citations_shard1_replica1/data/index/write.lock This cloud has 4 nodes. The collection has two shards with two replicas per shard. They are all hosted in a google cloud environment. So if the delete deleted the replica why would it then hold a lock? We want to understand this. We are using Solr 6.2.0 -- This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith. Click http://www.emdgroup.com/disclaimer to access the German, French, Spanish and Portuguese versions of this disclaimer.