Double writing / double reading gets messy because it requires an awful lot of work to ensure it’s correct as you move data around.
Just fixing it during compaction or scrub doesnt require app changes, just disable repair, start rewriting the sstables, enable repair again when you’re done. Especially if 18352 gives you a cassandra-internal “is this a legal timestamp” predicate, that seems way easier than changing all the app logic. > On Jul 15, 2026, at 8:21 AM, Abe Ratnofsky <[email protected]> wrote: > > For recovery, I’d recommend approaching this as a regular table migration, > where you start writing to a new table and backfill using > cassandra-analytics, filtering out all cells with WRITETIME in the distant > future. > > Freeze your backups so they don’t get purged. You might be able to recover > some overwritten values from backup SSTables if they weren’t compacted yet. > > On Wed, Jul 15, 2026, at 8:16 AM, Jeff Jirsa wrote: >> It’s not that it’s not running, it’s that ntpd in startup phases can >> have big steps in some configs. Also alert on large clock drift. If you >> run cassandra, you need alerts on ntpd >> >>> On Jul 15, 2026, at 8:13 AM, Yakir Gibraltar <[email protected]> wrote: >>> >>> Configure Cassandra to start after ntpd service , easy on systemd. >>> >>> Thank you, Yakir Gibraltar >>> >>> On Wed, Jul 15, 2026, 10:47 Panagiotis Melidis via user >>> <[email protected]> wrote: >>>> Hi all,____ >>>> __ __ >>>> I'm looking for guidance on a scenario involving NTP clock skew and its >>>> impact on Cassandra data.____ >>>> __ __ >>>> If an NTP malfunction causes the system clock on multiple Cassandra nodes >>>> to jump far into the future (e.g., 100 years), any writes processed during >>>> that window will carry far-future timestamps. Once the clock is corrected, >>>> these entries effectively become permanent and shadow any new write/delete >>>> in the affected partition keys.____ >>>> __ __ >>>> The only viable recovery approach at scale I've been able to identify is >>>> truncating the affected tables.____ >>>> 1. Are there any other known viable recovery procedures that don't require >>>> a full truncate?____ >>>> 2. Are there any known existing or planned preventive measures in >>>> Cassandra to guard against such scenarios?____ >>>> __ __ >>>> Any pointers to prior discussions, tooling, or patches would be greatly >>>> appreciated.____ >>>> __ __ >>>> Thanks.____
