[
https://issues.apache.org/jira/browse/KAFKA-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-16221:
------------------------------------
Description:
https://issues.apache.org/jira/browse/KAFKA-14831 fixed a producer bug about
internal TX state transition and the producer is now throwing an
IllegalStateException in situations it did swallow an internal error before.
This change surfaces a bug in Kafka Streams: Kafka Streams calls
`abortTransaction()` blindly when a task is closed dirty, even if the Producer
is already in an internal fatal state. However, if the Producer is in a fatal
state, Kafka Streams should skip `abortTransaction` and only `close()` the
Producer when closing a task dirty.
The bug is surfaced after `commitTransaction()` did timeout and another error
happens leading to the call to `abortTransaction()` – Kafka Streams does not
track right now if a commit-TX is in progress.
If the Producer throws an IllegalStateException on `abortTransaction()` Kafka
Streams treats this exception ("correctly") as fatal, and StreamsThread dies.
However, Kafka Streams is actually in a state in which it can recover from, and
thus should not let StreamThread die by carry forward (by not calling
`abortTransaction()` and moving forward with the dirty close of the task).
was:
https://issues.apache.org/jira/browse/KAFKA-14831 fixed a producer bug about
internal TX state transition and the producer is now throwing an
IllegalStateException in situations it did swallow an internal error before.
This change surfaces a bug in Kafka Streams: Kafka Streams calls
`abortTransaction()` blindly when a task is closed dirty, even if the Producer
is already in an internal fatal state. However, if the Producer is in a fatal
state, Kafka Streams should skip `abortTransaction` and only `close()` the
Producer when closing a task dirty.
If the Producer throws an IllegalStateException on `abortTransaction()` Kafka
Streams treats this exception ("correctly") as fatal, and StreamsThread dies.
However, Kafka Streams is actually in a state in which it can recover from, and
thus should not let StreamThread die by carry forward (by not calling
`abortTransaction()` and moving forward with the dirty close of the task).
> IllegalStateException from Producer
> -----------------------------------
>
> Key: KAFKA-16221
> URL: https://issues.apache.org/jira/browse/KAFKA-16221
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.6.0
> Reporter: Matthias J. Sax
> Priority: Critical
>
> https://issues.apache.org/jira/browse/KAFKA-14831 fixed a producer bug about
> internal TX state transition and the producer is now throwing an
> IllegalStateException in situations it did swallow an internal error before.
> This change surfaces a bug in Kafka Streams: Kafka Streams calls
> `abortTransaction()` blindly when a task is closed dirty, even if the
> Producer is already in an internal fatal state. However, if the Producer is
> in a fatal state, Kafka Streams should skip `abortTransaction` and only
> `close()` the Producer when closing a task dirty.
> The bug is surfaced after `commitTransaction()` did timeout and another error
> happens leading to the call to `abortTransaction()` – Kafka Streams does not
> track right now if a commit-TX is in progress.
> If the Producer throws an IllegalStateException on `abortTransaction()` Kafka
> Streams treats this exception ("correctly") as fatal, and StreamsThread dies.
> However, Kafka Streams is actually in a state in which it can recover from,
> and thus should not let StreamThread die by carry forward (by not calling
> `abortTransaction()` and moving forward with the dirty close of the task).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
