Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2025-03-08 Thread Jon Haddad
I really like the data directories and replication configuration. I think it makes a ton of sense to put it in the yaml, but we should probably yaml it all, and not nest JSON :), and we can probably simplify it a little with a file uri scheme, something like this: data_file_locations: disk:

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2025-03-08 Thread Štefan Miklošovič
That is cool but this still does not show / explain how it would look like when it comes to dependencies needed for actually talking to storages like s3. Maybe I am missing something here and please explain when I am mistaken but If I understand that correctly, for talking to s3 we would need to u

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2025-03-08 Thread Joseph Lynch
Jon, I like where you are headed with that, just brainstorming out what the end interface might look like (might be getting a bit ahead of things talking about directories if we don't even have files implemented yet). What do folks think about pairing data_file_locations (fka data_file_directories)

Re: [UPDATE] CEP-37

2025-03-08 Thread Jaydeep Chovatia
Hi David, Thanks for the kind words! >Is there a goal in this CEP to make automated repair work during rolling upgrades, when multiple versions exist in the cluster? We debated a lot on this over ASF Slack (#cassandra-repair-scheduling-cep37). The summary is that, ideally, we want to have a repai

Re: [UPDATE] CEP-37

2025-03-08 Thread Dave Herrington
Jaydeep, Thank you for your excellent efforts on this mission-critical feature. The stated goals of CEP-37 are noble and stand to make valuable improvements for cluster operations. I look forward to testing these new capabilities. My apologies up-front if you’ve already answered these questions

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2025-03-08 Thread Jon Haddad
Taking this a step further, this opens up is a different way of bootstrapping new nodes, by using the object store's native copy commands. This is something that's impossible when just using the filesystem mount. I think Joey mentioned something along these lines to me several years ago, maybe at t

Re: [RELEASE] Apache Cassandra Sidecar 0.1.0 released

2025-03-08 Thread Paulo Motta
Damn.. listing name sucks. Also Saranya and whoever else was involved in this epic milestone! Taking the occasion, happy international women's day to all Cassandrettes 🌹 On Sat, 8 Mar 2025 at 11:24 Paulo Motta wrote: > Congrats on this release! 🍾👏👏 Awesome work to all involved, specially > but

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2025-03-08 Thread Jon Haddad
Thanks Jordan and Joey for the additional info. One thing I'd like to clarify - what I'm mostly after is 100% of my data on object store, local disk acting as a LRU cache, although there's also a case for the mirror. What I see so far are three high level ways of running this: 1. Mirror Mode Th

Re: [RELEASE] Apache Cassandra Sidecar 0.1.0 released

2025-03-08 Thread Paulo Motta
Congrats on this release! 🍾👏👏 Awesome work to all involved, specially but not limited to Dinesh, Francisco, Bernado et al 😊 On Fri, 7 Mar 2025 at 16:48 Patrick McFadin wrote: > We happen to have some very informed engineers discussing Sidecar and > the many things you can do with it at Cassandra

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2025-03-08 Thread Joseph Lynch
Great discussion - I agree strongly with Jon's points, giving operators this option will make many operator's lives easier. Even if you still have to have 100% disk space to meet performance requirements, that's still much more efficient than you can run C* with just disks (as you need to leave hea