[
https://issues.apache.org/jira/browse/KAFKA-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864284#comment-17864284
]
Andrew Schofield commented on KAFKA-17093:
------------------------------------------
[~tkalmijn] I think there's a path forward here. The problem you have is that
you're struggling to deal with unknown numbers of control records when you're
trying to read all of the records. I think the following works:
* Call KafkaConsumer.seekToEnd() and then KafkaConsumer.position() - this will
skip over the control records and position the consumer right after all of the
records which it can read.
* Seek back to wherever you want to consume from.
* Consume using KafkaConsumer.poll(). When the offset reaches the end offset,
or KafkaConsumer.position() reaches that point, you've consumed all of the
records.
I've added a new version of the test app.
> KafkaConsumer.seekToEnd should return LSO
> ------------------------------------------
>
> Key: KAFKA-17093
> URL: https://issues.apache.org/jira/browse/KAFKA-17093
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.6.1
> Environment: Ubuntu, IntelliJ, Scala "org.apache.kafka" %
> "kafka-clients" % "3.6.1"
> Reporter: Tom Kalmijn
> Assignee: Andrew Schofield
> Priority: Major
> Attachments: Kafka17093-v2.java, Kafka17093-v3.java, Kafka17093.java
>
>
>
> Expected
> When using a transactional producer then the method
> KafkaConsumer.seekToEnd(...) of a consumer configured with isolation level
> "read_committed" should return the LSO.
> Observed
> The offset returned is always the actual last offset of the partition, which
> is not the LSO if the latest offsets are occupied by transaction markers.
> Also see this Slack thread:
> https://confluentcommunity.slack.com/archives/C499EFQS0/p1720088282557559
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
