Thanks for the idea. I tried it, and the state for the bad node, even
after an orderly shutdown, is still "active" in clusterstate.json. I see
this in the logs on restart:
[28 Jan 2014 18:25:29] [RecoveryThread] ERROR
(org.apache.solr.common.SolrException) - Error while trying to recover.
core=marin:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
I was asked to wait on state recovering for truffle-solr-4:8983_solr but I
still do not see the requested state. I see state: active live:true
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:198)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:342)
at
org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:219)
-Greg
On Tue, Jan 28, 2014 at 9:53 AM, Shawn Heisey <s...@elyograg.org> wrote:
> On 1/28/2014 10:31 AM, Greg Preston wrote:
>
>> ** Using solrcloud 4.4.0 **
>>
>> I had to kill a running solrcloud node. There is still a replica for that
>> shard, so everything is functional. We've done some indexing while the
>> node was killed.
>>
>> I'd like to bring back up the downed node and have it resync from the
>> other
>> replica. But when I restart the downed node, it joins back up as active
>> immediately, and doesn't resync. I even wiped the data directory on the
>> downed node, hoping that would force it to sync on restart, but it
>> doesn't.
>>
>> I'm assuming this is related to the state still being listed as active in
>> clusterstate.json for the downed node? Since it comes back as active,
>> it's
>> serving queries and giving old results.
>>
>> How can I force this node to do a recovery on restart?
>>
>
> This might be completely wrong, but hopefully it will help you: Perhaps a
> graceful stop of that node will result in the proper clusterstate so it
> will work the next time it's started? That may already be what you've done,
> so this may not help at all ... but you did say "kill" which might mean
> that it wasn't a clean shutdown of Solr.
>
> Thanks,
> Shawn
>
>
