[
https://issues.apache.org/jira/browse/KAFKA-7525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954472#comment-16954472
]
Tobias Neubert commented on KAFKA-7525:
---------------------------------------
Encountered the same behaviour with the IllegalStateException in verison 2.2.0.
Is this "clear bug" fixed in 2.3.0?
> Handling corrupt records
> ------------------------
>
> Key: KAFKA-7525
> URL: https://issues.apache.org/jira/browse/KAFKA-7525
> Project: Kafka
> Issue Type: Improvement
> Components: consumer, core
> Affects Versions: 1.1.0
> Reporter: Katarzyna Solnica
> Priority: Major
>
> When Java consumer encounters a corrupt record on a partition it reads from,
> it throws:
> {code:java}
> org.apache.kafka.common.KafkaException: Received exception when fetching the
> next record from XYZ. If needed, please seek past the record to continue
> consumption.
> at
> org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1125)
> at
> org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1400(Fetcher.java:993)
> at
> org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:527)
> at
> org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:488)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1155)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
> (...)
> Caused by: org.apache.kafka.common.errors.CorruptRecordException: Record size
> is less than the minimum record overhead (14){code}
> or:
> {code:java}
> java.lang.IllegalStateException: Unexpected error code 2 while fetching data
> at
> org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:936)
> at
> org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:485)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1155)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
> (...){code}
> 1. Could you consider throwing CorruptRecordException from
> parseCompletedFetch() when error == Errors.CORRUPT_MESSAGE?
> 2. Seeking past the corrupt record means losing data. I've noticed that the
> record is often correct on a follower ISR, and manual change of the partition
> leader to the follower node solves the issue in case partition is used by a
> single consumer group. Couldn't Kafka server discover such situations and
> recover corrupt records from logs available on other ISRs somehow?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
