[
https://issues.apache.org/jira/browse/KAFKA-14727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Egerton updated KAFKA-14727:
----------------------------------
Affects Version/s: 3.3.2
3.3.1
3.3.0
> Connect EOS mode should periodically call task commit
> -----------------------------------------------------
>
> Key: KAFKA-14727
> URL: https://issues.apache.org/jira/browse/KAFKA-14727
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 3.3.0, 3.4.0, 3.3.1, 3.3.2
> Reporter: Greg Harris
> Assignee: Greg Harris
> Priority: Major
>
> In non-EOS mode, there is a background thread which periodically commits
> offsets for a task. If this thread does not have resources to flush on the
> framework side (records, or offsets) it still calls the task's commit()
> method to update the internal state of the task.
> In EOS mode, there is no background thread, and all offset commits are
> performed on the main task thread in response to sending records to Kafka.
> This has the effect of only triggering the task's commit() method when there
> are records to send to Kafka, which is different than non-EOS mode.
> In order to bring the two modes into better alignment, and allow tasks
> reliant on the non-EOS empty commit() behavior to work in EOS mode
> out-of-the-box, EOS mode should provide offset commits periodically for tasks
> which do not produce records.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
