[
https://issues.apache.org/jira/browse/KAFKA-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17878411#comment-17878411
]
Anh Tuan Nguyen commented on KAFKA-17101:
-----------------------------------------
I'm facing same issue with mirror maker 3.6.0 (issue was already reported by my
customers in older releases).
I don't use Kraft, it's just mirror maker to replicate between 2 geo redundancy
Kafka clusters.
As [~kaushik srinivas] mentioned, there's no intervention to these internal
topics, and issue is not easily reproduced too.
We're having to maintain a script to update topics config from delete to
compact as workaround.
> Mirror maker internal topics cleanup policy changes to 'delete' from
> 'compact'
> -------------------------------------------------------------------------------
>
> Key: KAFKA-17101
> URL: https://issues.apache.org/jira/browse/KAFKA-17101
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 3.4.1, 3.5.1, 3.6.1
> Reporter: kaushik srinivas
> Priority: Major
>
> Scenario/Setup details
> Kafka cluster 1: 3 replicas
> Kafka cluster 2: 3 replicas
> MM1 moving data from cluster 1 to cluster 2
> MM2 moving data from cluster 2 to cluster 1
> Sometimes with a reboot of the kafka cluster 1 and MM1 instance, we observe
> MM failing to come up with below exception,
> {code:java}
> {"message":"DistributedHerder-connect-1-1 -
> org.apache.kafka.connect.runtime.distributed.DistributedHerder - [Worker
> clientId=connect-1, groupId=site1-mm2] Uncaught exception in herder work
> thread, exiting: "}}
> org.apache.kafka.common.config.ConfigException: Topic
> 'mm2-offsets.site1.internal' supplied via the 'offset.storage.topic' property
> is required to have 'cleanup.policy=compact' to guarantee consistency and
> durability of source connector offsets, but found the topic currently has
> 'cleanup.policy=delete'. Continuing would likely result in eventually losing
> source connector offsets and problems restarting this Connect cluster in the
> future. Change the 'offset.storage.topic' property in the Connect worker
> configurations to use a topic with 'cleanup.policy=compact'. {code}
> Once the topic is altered with cleanup policy of compact. MM works just fine.
> This is happening on our setups sporadically and across varieties of
> scenarios. Not been successful in identifying the exact reproduction steps as
> of now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
