Re: TTL and disk space releasing

2021-10-07 Thread Michel Barret
Hi it's my point as we have only one way to insert data it seems be cool to set it by code to allow change it by configuration if need. I use another way to clean my cluster : I down a node, delete bad sstables manually, up & repair the node. And apply it node by node. Now the nodes don't con

Re: TTL and disk space releasing

2021-10-06 Thread Jeff Jirsa
I think this is a bit extreme. If you know that 100% of all queries that write to the table include a TTL, not having a TTL on the table is just fine. You just need to ensure that you always write correctly. On Wed, Oct 6, 2021 at 8:57 AM Bowen Song wrote: > TWCS without a table TTL is unlikely

Re: TTL and disk space releasing

2021-10-06 Thread Bowen Song
TWCS without a table TTL is unlikely to work correctly, and adding the table TTL retrospectively alone is also unlikely to fix the existing issue. You may need to add the table default TTL and update all existing data to reflect the TTL change, and then trigger a major compaction to update the

Re: TTL and disk space releasing

2021-10-06 Thread Michel Barret
Hi, it's not set before. I set it to ensure all data have a ttl. Thanks for your help. Le 06/10/2021 à 13:47, Bowen Song a écrit : What is the the table's default TTL? (Note: it may be different than the TTL of the data in the table) On 06/10/2021 09:42, Michel Barret wrote: Hello, I try to

Re: TTL and disk space releasing

2021-10-06 Thread Michel Barret
Thank you for your pointers. sstablemetadata seem's explain that we have data without ttl (= 0) I don't know how can appears in ours system. - I replace our ttl by query by the default ttl. - I reduce the gc grace seconds to one day - I apply the unchecked_tombstone_compaction (on 31 days of dat

Re: TTL and disk space releasing

2021-10-06 Thread Bowen Song
What is the the table's default TTL? (Note: it may be different than the TTL of the data in the table) On 06/10/2021 09:42, Michel Barret wrote: Hello, I try to use cassandra (3.11.5) with 8 nodes (in single datacenter). I use one simple table, all data are inserted with 31 days TTL (the data

Re: TTL and disk space releasing

2021-10-06 Thread Paul Chandler
Hi Michael, I have had similar problems in the past, and found this Last Pickle post very useful: https://thelastpickle.com/blog/2016/12/08/TWCS-part1.html This should help you pinpoint what is stopping the SSTables being deleted. Assuming you are never manually deleting records from the table

TTL and disk space releasing

2021-10-06 Thread Michel Barret
Hello, I try to use cassandra (3.11.5) with 8 nodes (in single datacenter). I use one simple table, all data are inserted with 31 days TTL (the data are never updated). I use the TWCS strategy with: - 'compaction_window_size': '24' - 'compaction_window_unit': 'HOURS' - 'max_threshold': '32' -