Re: CDC Tools

2020-05-27 Thread Dor Laor
If it's helpful, IMO, the approach Cassandra needs to take isn't by tracking the individual node commit log and putting the burden on the client. At Scylla, we had the 'opportunity' to be a late comer and see what approach Cassadnra took and what DynamoDB streams took. We've implemented CDC as a r

Re: CDC Tools

2020-05-27 Thread Erick Ramirez
> > I have looked at DataStax CDC but I think it works only for DSE ! > Yes, thanks for the correction. I just got confirmation myself -- the Kafka-Cassandra connector works with OSS C* but the CDC connector relies on a DSE feature that's not yet available in OSS C*. Cheers!

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Nitan Kainth
Yeah, I meant the down node can’t participate in repairs Regards, Nitan Cell: 510 449 9629 > On May 27, 2020, at 2:09 PM, Leon Zaruvinsky wrote: > >  > Yep, Jeff is right, the intention would be to run a repair limited to the > available nodes. > >> On Wed, May 27, 2020 at 2:59 PM Jeff Jirs

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Leon Zaruvinsky
Yep, Jeff is right, the intention would be to run a repair limited to the available nodes. On Wed, May 27, 2020 at 2:59 PM Jeff Jirsa wrote: > The "-hosts " flag tells cassandra to only compare trees/run repair on the > hosts you specify, so if you have 3 replicas, but 1 replica is down, you > c

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Jeff Jirsa
The "-hosts " flag tells cassandra to only compare trees/run repair on the hosts you specify, so if you have 3 replicas, but 1 replica is down, you can provide -hosts with the other two, and it will make sure those two are in sync (via merkle trees, etc), but ignore the third. On Wed, May 27, 20

Re: Generating evenly distributed tokens for vnodes

2020-05-27 Thread Kornel Pal
As I understand, the previous discussion is about using allocate_tokens_for_keyspace for allocating tokens for most of the nodes. On the other hand, I am proposing to generate all the tokens for all the nodes using a Python script. This seems to result in perfectly even token ownership distrib

Re: Generating evenly distributed tokens for vnodes

2020-05-27 Thread Elliott Sims
There's also a slightly older mailing list discussion on this subject that goes into detail on this sort of strategy: https://www.mail-archive.com/user@cassandra.apache.org/msg60006.html I've been approximately following it, repeating steps 3-6 for the first host in each "rack(replica, since I hav

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Nitan Kainth
Jeff, If Cassandra is down how will it generate merkle tree to compare? Regards, Nitan Cell: 510 449 9629 > On May 27, 2020, at 11:15 AM, Jeff Jirsa wrote: > >  > You definitely can repair with a node down by passing `-hosts specific_hosts` > >> On Wed, May 27, 2020 at 9:06 AM Nitan Kainth

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Jeff Jirsa
You definitely can repair with a node down by passing `-hosts specific_hosts` On Wed, May 27, 2020 at 9:06 AM Nitan Kainth wrote: > I didn't get you Leon, > > But, the simple thing is just to follow the steps and you will be fine. > You can't run the repair if the node is down. > > On Wed, May 2

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Nitan Kainth
I didn't get you Leon, But, the simple thing is just to follow the steps and you will be fine. You can't run the repair if the node is down. On Wed, May 27, 2020 at 10:34 AM Leon Zaruvinsky wrote: > Hey Jeff/Nitan, > > 1) this concern should not be a problem if the repair happens before the > c

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Leon Zaruvinsky
Hey Jeff/Nitan, 1) this concern should not be a problem if the repair happens before the corrupted node is brought back online, right? 2) in this case, is option (3) equivalent to replacing the node? where we repair the two live nodes and then bring up the third node with no data Leon On Tue, Ma

Re: any risks with changing replication factor on live production cluster without downtime and service interruption?

2020-05-27 Thread Leena Ghatpande
Nothing complex. All we do is perform the read for x number of retries (configurable parameter) and if it fails , flag an alert But agree with the solution that Jeff provided and would use the approach. Thanks for all the responses. From: Reid Pinchback Sent:

Generating evenly distributed tokens for vnodes

2020-05-27 Thread Kornel Pal
Hi, Generating ideal tokens for single-token datacenters is well understood and documented, but there is much less information available on generating tokens with even ownership distribution when using vnodes. The best description I could find on token generation for vnodes is https://thelast

Re: CDC Tools

2020-05-27 Thread Ahmed Eljami
Hi Erick, I have looked at DataStax CDC but I think it works only for DSE ! Maybe it will be open-sourced like other drivers :) Cheers. Le mer. 27 mai 2020 à 04:42, Erick Ramirez a écrit : > I think I recall seeing you post the same question on ASF Slack yesterday. > > I haven't used Debezium s