Re: How to replace a dead *seed* node while keeping quorum

2012-11-07 Thread Ron Siemens
I have an update on this. I witnessed this same split ring problem, this time while doing a rolling upgrade from 1.1.4 to 1.1.6. I found an easier workaround than modifying configs and restarting. I found that by explicitly specifying the same token on the commandline using "-Dcassandra.repl

Re: How to replace a dead *seed* node while keeping quorum

2012-10-10 Thread Ron Siemens
I witnessed the same behavior as reported by Edward and James. Removing the host from its own seed list does not solve the problem. Removing it from config of all nodes and restarting each, then restarting the failed node worked. Ron On Sep 12, 2012, at 4:42 PM, Edward Sargisson wrote: > I'

Re: How to replace a dead *seed* node while keeping quorum

2012-09-12 Thread Edward Sargisson
I'm reposting my colleague's reply to Rob to the list (with James' permission) in case others are interested. I'll add to James' post below to say I don't believe we saw the message that that slice of code would have printed. " Hey Rob, Ed's AWOL right now and I'm not onu@c.a.o, but I can t

Re: How to replace a dead *seed* node while keeping quorum

2012-09-12 Thread Mohit Anchlia
How can this be resolved in this case? On Wed, Sep 12, 2012 at 3:53 PM, Rob Coli wrote: > On Tue, Sep 11, 2012 at 4:21 PM, Edward Sargisson > wrote: > > If the downed node is a seed node then neither of the replace a dead node > > procedures work (-Dcassandra.replace_token and taking initial_to

Re: How to replace a dead *seed* node while keeping quorum

2012-09-12 Thread Rob Coli
On Tue, Sep 11, 2012 at 4:21 PM, Edward Sargisson wrote: > If the downed node is a seed node then neither of the replace a dead node > procedures work (-Dcassandra.replace_token and taking initial_token-1). The > ring remains split. > [...] > In other words, if the host name is on the seeds list t

How to replace a dead *seed* node while keeping quorum

2012-09-11 Thread Edward Sargisson
Hi all, We just ran into an interesting and unexpected situation with restarting a downed node. If the downed node is a seed node then neither of the replace a dead node procedures work (-Dcassandra.replace_token and taking initial_token-1). The ring remains split. The host is listed as a see