[
https://issues.apache.org/jira/browse/KAFKA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Cadonna updated KAFKA-13549:
----------------------------------
Fix Version/s: 3.2.0
> Add "delete interval" config
> ----------------------------
>
> Key: KAFKA-13549
> URL: https://issues.apache.org/jira/browse/KAFKA-13549
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Nicholas Telford
> Priority: Major
> Labels: kip
> Fix For: 3.2.0
>
>
> KIP-811:
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-811%3A+Add+config+min.repartition.purge.interval.ms+to+Kafka+Streams]
> Kafka Streams uses "delete record" requests to aggressively purge data from
> repartition topics. Those request are sent each time we commit.
> For at-least-once with a default commit interval of 30 seconds, this works
> fine. However, for exactly-once with a default commit interval of 100ms, it's
> very aggressive. The main issue is broker side, because the broker logs every
> "delete record" request, and thus broker logs are spammed if EOS is enabled.
> We should consider to add a new config (eg `delete.record.interval.ms` or
> similar) to have a dedicated config for "delete record" requests, to decouple
> it from the commit interval config and allow to purge data less aggressively,
> even if the commit interval is small to avoid the broker side log spamming.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
