[
https://issues.apache.org/jira/browse/KAFKA-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-20036.
------------------------------------
Fix Version/s: 4.3.0
Resolution: Fixed
> Handle LogCleaner segment overflow caused by compression level changes
> ----------------------------------------------------------------------
>
> Key: KAFKA-20036
> URL: https://issues.apache.org/jira/browse/KAFKA-20036
> Project: Kafka
> Issue Type: Bug
> Reporter: Chia-Ping Tsai
> Assignee: Ken Huang
> Priority: Major
> Fix For: 4.3.0
>
>
> from: [https://github.com/apache/kafka/pull/15516#discussion_r2649600413]
> The {{LogCleaner}} can fail with {{java.lang.IllegalArgumentException}} when
> the compacted data exceeds the segment size limit.
> This issue occurs because the cleaner rebuilds batches using the default
> compression level. If the data was originally ingested with a higher
> compression level, the re-compressed batch size increases, causing the
> cleaner's size estimation to be inaccurate and leading to an overflow.
> To address this without major architectural changes (like splitting segments
> during cleaning), we should introduce a safety margin or adaptive logic for
> partitions that encounter this overflow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
