Re: [DISCUSS] Snapshots outside of Cassandra data directory

2025-01-12 Thread Štefan Miklošovič
C) Let's just enable backuping to a local filesystem. To make things simpler and more user-friendly, it would be stored the same way (in the target destination) Sidecar would upload it, so when people decide to start to use Sidecar and incorporate it into their deployments / workflows, these backu

Re: [DISCUSS] Snapshots outside of Cassandra data directory

2025-01-12 Thread Jon Haddad
Are you proposing that we manage backups in the DB instead of Sidecar, or that we have the same functionality in both C* proper and the sidecar? Or that we ship C* with backups to a local filesystem only? Where should the line be on what goes into sidecar and what goes into C* proper? Jon On

Re: [DISCUSS] Snapshots outside of Cassandra data directory

2025-01-12 Thread Štefan Miklošovič
Oh yeah I knew Sidecar will be mentioned, let's dive into that. Sidecar has a lot of endpoints / functionality, backup / restore is just part of that. What I proposed has also thes advantages: 1) Every time you go to upload to some cloud storage provider, you need to add all the dependencies to

Re: [DISCUSS] Snapshots outside of Cassandra data directory

2025-01-12 Thread Jon Haddad
Sound like part of a backup strategy.Probably worth chiming in on the sidecar issue: https://issues.apache.org/jira/browse/CASSSIDECAR-148. IIRC, Medusa and Tablesnap both uploaded a manifest and don't upload multiple copies of the same SSTables. I think this should definitely be part of our

[DISCUSS] Snapshots outside of Cassandra data directory

2025-01-12 Thread Štefan Miklošovič
Hi, I would like to run this through ML to gather feedback as we are contemplating about making this happen. Currently, snapshots are just hardlinks located in a snapshot directory to live data directory. That is super handy as it occupies virtually zero disk space etc (as long as underlying SSTa