Re: Bringing a dead node back up after fixing hardware issues

2012-07-30 Thread Eran Chinthaka Withana
Thanks Brandon. Thanks, Eran Chinthaka Withana On Thu, Jul 26, 2012 at 12:46 PM, Brandon Williams wrote: > On Wed, Jul 25, 2012 at 6:16 PM, Eran Chinthaka Withana > wrote: > > > Alright, lets assume I want to go on this route. I have RF=2 in the data > > center and I believe I need at least R

Re: Bringing a dead node back up after fixing hardware issues

2012-07-26 Thread Brandon Williams
On Wed, Jul 25, 2012 at 6:16 PM, Eran Chinthaka Withana wrote: > Alright, lets assume I want to go on this route. I have RF=2 in the data > center and I believe I need at least RF=3 to set the replication to > LOCAL_QUORUM and hide the node failures. But if I increase the RF to 3 now > then won't

Re: Bringing a dead node back up after fixing hardware issues

2012-07-25 Thread Eran Chinthaka Withana
Hi Brandon, > Increasing CL is tricky for us for now, as our RF on that datacenter is 2 > > and CL is set to ONE. If we make the CL to be LOCAL_QUORUM, then, if a > node > > goes down we will have trouble. I will try to increase the RF to 3 in > that > > data center and set the CL to LOCAL_QUORUM

Re: Bringing a dead node back up after fixing hardware issues

2012-07-24 Thread Brandon Williams
On Mon, Jul 23, 2012 at 10:24 PM, Eran Chinthaka Withana wrote: > Thanks Brandon for the answer (and I didn't know driftx = Brandon Williams. > Thanks for your awesome support in Cassandra IRC) Thanks :) > Increasing CL is tricky for us for now, as our RF on that datacenter is 2 > and CL is set

Re: Bringing a dead node back up after fixing hardware issues

2012-07-23 Thread Eran Chinthaka Withana
Thanks Brandon for the answer (and I didn't know driftx = Brandon Williams. Thanks for your awesome support in Cassandra IRC) Increasing CL is tricky for us for now, as our RF on that datacenter is 2 and CL is set to ONE. If we make the CL to be LOCAL_QUORUM, then, if a node goes down we will have

Re: Bringing a dead node back up after fixing hardware issues

2012-07-23 Thread Brandon Williams
On Mon, Jul 23, 2012 at 6:26 PM, Eran Chinthaka Withana wrote: > Method 1: I copied the data from all the nodes in that data center, into the > repaired node, and brought it back up. But because of the rate of updates > happening, the read misses started going up. That's not really a good method

Bringing a dead node back up after fixing hardware issues

2012-07-23 Thread Eran Chinthaka Withana
Hi, In my cluster, one of the nodes went down (due to a hardware failure). We managed to get it fixed in couple of days. But it seems its harder to bring this same node back into cluster without creating read misses. Here is what I did. Method 1: I copied the data from all the nodes in that data