vasya b created CASSANDRA-21245:
-----------------------------------
Summary: Uncompressed size is being used for compressed tables
Key: CASSANDRA-21245
URL: https://issues.apache.org/jira/browse/CASSANDRA-21245
Project: Apache Cassandra
Issue Type: Bug
Reporter: vasya b
Attachments: go.mod, main.go
Using a compressed table can lead to a state where uncompressed table size is
bigger
than the whole volume size, while it shouldn't be a problem it leads to
compaction poblems, as other compaction task are being denied.
f.e. one can easily create a compressed table with
{code:java}
CREATE TABLE IF NOT EXISTS ` + keyspace + `.` + table + ` (
pk bigint,
data text,
PRIMARY KEY (pk)
) WITH compression = {'class': 'DeflateCompressor'}
{code}
(or any other compression)
Insert a bunch of data which is going to be compressed quite well (f.e. a bunch
of zeros or any other single char * 1024 *1024) * N where N is some big number
one would expect that such table will be compacted without any problems.
but one will get:
{code:java}
WARN [CompactionExecutor:8] 2026-03-20 14:25:17,256 CompactionTask.java:434 -
Not enough space for compaction (78c5c7c0-244f-11f1-9e8a-9937f6f9033a) of
zerotohero.bulk_data, estimated sstables = 1, expected write size = 2686214
ERROR [CompactionExecutor:8] 2026-03-20 14:25:17,256
JVMStabilityInspector.java:70 - Exception in thread
Thread[CompactionExecutor:8,5,CompactionExecutor]
java.lang.RuntimeException: Not enough space for compaction
(78c5c7c0-244f-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data, estimated
sstables = 1, expected write size = 2686214
at
org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:436)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:148)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:26)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:94)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:374)
at
org.apache.cassandra.concurrent.FutureTask$3.call(FutureTask.java:141)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:840)
WARN [CompactionExecutor:9] 2026-03-20 15:21:27,534 CompactionTask.java:440 -
Not enough space for compaction 51834c70-2457-11f1-9e8a-9937f6f9033a,
71.31226MiB estimated. Reducing scope.
DEBUG [CompactionExecutor:8] 2026-03-20 15:25:27,447 Directories.java:550 -
FileStore /mnt/cassandra (/dev/sdb1) has 36221336371 bytes available, checking
if we can write 95979501107 bytes
WARN [CompactionExecutor:8] 2026-03-20 15:25:27,447 Directories.java:553 -
FileStore /mnt/cassandra (/dev/sdb1) has only 33.73 GiB available, but 89.39
GiB is needed
WARN [CompactionExecutor:8] 2026-03-20 15:25:27,447 CompactionTask.java:434 -
Not enough space for compaction (e08281c0-2457-11f1-9e8a-9937f6f9033a) of
zerotohero.bulk_data_zstd, estimated sstables = 1, expected write size = 2195528
ERROR [CompactionExecutor:8] 2026-03-20 15:25:27,447
JVMStabilityInspector.java:70 - Exception in thread
Thread[CompactionExecutor:8,5,CompactionExecutor]
java.lang.RuntimeException: Not enough space for compaction
(e08281c0-2457-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data_zstd, estimated
sstables = 1, expected write size = 2195528
at
org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:436)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:148)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:26)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:94)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:374)
at
org.apache.cassandra.concurrent.FutureTask$3.call(FutureTask.java:141)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:840)
DEBUG [CompactionExecutor:8] 2026-03-20 15:25:27,447 HeapUtils.java:133 - Heap
dump creation on uncaught exceptions is disabled.
ERROR [CompactionExecutor:12] 2026-03-20 16:09:13,125
JVMStabilityInspector.java:70 - Exception in thread
Thread[CompactionExecutor:12,5,CompactionExecutor]
java.lang.RuntimeException: Not enough space for compaction
(fd9ba330-245d-11f1-9e8a-9937f6f9033a) of zerotohero.bulk_data_deflate,
estimated sstables = 1, expected write size = 2200455
at
org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:436)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:148)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:26)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:94)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:374)
at
org.apache.cassandra.concurrent.FutureTask$3.call(FutureTask.java:141)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:840)
DEBUG [CompactionExecutor:12] 2026-03-20 16:09:13,126 HeapUtils.java:133 - Heap
dump creation on uncaught exceptions is disabled. {code}
while there is enough space:
{code:java}
root@zerotohero:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 49G 4.5G 42G 10% /mnt/cassandra
root@zerotohero:~# nodetool compactionstats -H
concurrent compactors 2
pending tasks 31
zerotohero bulk_data 31
compactions completed 5009
data compacted 1.1 TiB
compactions aborted 174
compactions reduced 1
sstables dropped from compaction 51
15 minute rate 5.33/minute
mean rate 1226.87/hour
compaction throughput (MiB/s) 64.0
id compaction type keyspace table
completed total unit progress
c73b9e00-2451-11f1-9e8a-9937f6f9033a Compaction zerotohero bulk_data 58.16
GiB 136.33 GiB bytes 42.66%
active compaction remaining time 0h20m50s {code}
simple reproducer in go included
cassandra version tested: 5.0.6
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
