Re: problem removing dead node from ring

2014-06-07 Thread Curious Patient
Hey all, OK I gave removing the downed node from the cassandra ring another try. To recap what's going on, this is what my ring looks like with nodetool status: [root@beta-new:~] #nodetool status Datacenter: datacenter1 === Status=Up/Down |/ State=Normal/Leaving/Joining/

Re: problem removing dead node from ring

2014-06-04 Thread Robert Coli
On Tue, Jun 3, 2014 at 9:03 PM, Matthew Allen wrote: > Thanks Robert, this makes perfect sense. Do you know if CASSANDRA-6961 > will be ported to 1.2.x ? > I just asked driftx, he said "not gonna happen." > And apologies if these appear to be dumb questions, but is a repair more > suitable th

Re: problem removing dead node from ring

2014-06-03 Thread Matthew Allen
| That would work, but until CASSANDRA-6961 [1] there is no way to prevent this node from having a long window where it may serve stale | reads at CLs below QUORUM, until the rebuild completes. Thanks Robert, this makes perfect sense. Do you know if CASSANDRA-6961 will be ported to 1.2.x ? And a

Re: problem removing dead node from ring

2014-06-03 Thread Robert Coli
On Tue, Jun 3, 2014 at 3:48 PM, Matthew Allen wrote: > Just out of curiosity, for a dead node, would it be possible to just > > - replace the node (no data in data/commit dirs), same IP Address, same > hostname. > - restore the cassandra.yaml (initial_token etc) > - set auto_bootstrap:false >

Re: problem removing dead node from ring

2014-06-03 Thread Matthew Allen
Just out of curiosity, for a dead node, would it be possible to just - replace the node (no data in data/commit dirs), same IP Address, same hostname. - restore the cassandra.yaml (initial_token etc) - set auto_bootstrap:false - start it up and then run a nodetool rebuild ? Or would the Host

Re: problem removing dead node from ring

2014-06-03 Thread Robert Coli
On Tue, Jun 3, 2014 at 11:03 AM, Curious Patient wrote: > In nodetool ring, if you're not using vnodes, only one token should be >> listed with both the IP of the old node and the status Down. >> If you are using vnodes, it's a comma delimited list in initial_token, >> which you can get from : >>

Re: problem removing dead node from ring

2014-06-03 Thread Curious Patient
> > In nodetool ring, if you're not using vnodes, only one token should be > listed with both the IP of the old node and the status Down. > If you are using vnodes, it's a comma delimited list in initial_token, > which you can get from : > nodetool info -T | grep Token | awk '{print $3}' | paste -s

Re: problem removing dead node from ring

2014-06-03 Thread Robert Coli
On Tue, Jun 3, 2014 at 10:53 AM, Curious Patient wrote: > I want to be sure I'm using the right token. > In nodetool ring, if you're not using vnodes, only one token should be listed with both the IP of the old node and the status Down. If you are using vnodes, it's a comma delimited list in in

Re: problem removing dead node from ring

2014-06-03 Thread Curious Patient
Hi Rob, If you are replacing an address, you need to use the identical > initial_token to the node you are replacing, not the token -1. Thanks, I hope that does the trick. Btw.. was my idea of how to get at the initial token of the missing/dead node correct? .i.e. nodetool ring | grep 10.10.1

Re: problem removing dead node from ring

2014-06-03 Thread Robert Coli
On Tue, Jun 3, 2014 at 8:41 AM, Curious Patient wrote: > I then started following this documentation on how to replace the node: > [ > http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_replace_node_t.html?scroll=task_ds_aks_15q_gk][1 > ] > ... > And set the initial to

problem removing dead node from ring

2014-06-03 Thread Curious Patient
One of the nodes in a cassandra cluster has died. I'm using cassandra 2.0.7 throughout. When I do a nodetool status this is what I see (real addresses have been replaced with fake 10 nets) [root@beta-new:/opt] #nodetool status Datacenter: datacenter1 === Statu