[
https://issues.apache.org/jira/browse/KAFKA-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrian C. updated KAFKA-19126:
------------------------------
Description:
We have observed during normal operations that messages/batches can be written
to disk with the same offset or baseOffset. This happens once in a while in
multiple of our clusters (hosted in Azure and AWS) which blocks follower
replication (error in attachment), causing ISR to shrink to the leader. In
version 3.7 and above it also blocks consumers clients from reading past the
offset.
As mitigation, we deciced not to upgrade from 3.6.1 and we're temporarily
setting min.isr.replicas to 1 until the record expires from the topic.
was:We have observed during normal operations that messages/batches can be
written to disk with the same offset or baseOffset. This happens once in a
while in multiple of our clusters (hosted in Azure and AWS) which blocks
follower replication (error in attachment), causing ISR to shrink to the
leader. In version 3.7 and above it also blocks consumers clients from reading
past the offset.
> Duplicate offsets written to log for different messages
> -------------------------------------------------------
>
> Key: KAFKA-19126
> URL: https://issues.apache.org/jira/browse/KAFKA-19126
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 3.6.1, 3.8.0, 3.7.1
> Reporter: Adrian C.
> Priority: Major
> Attachments: follower-error.log, log-dump-sample.txt
>
>
> We have observed during normal operations that messages/batches can be
> written to disk with the same offset or baseOffset. This happens once in a
> while in multiple of our clusters (hosted in Azure and AWS) which blocks
> follower replication (error in attachment), causing ISR to shrink to the
> leader. In version 3.7 and above it also blocks consumers clients from
> reading past the offset.
> As mitigation, we deciced not to upgrade from 3.6.1 and we're temporarily
> setting min.isr.replicas to 1 until the record expires from the topic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
