[
https://issues.apache.org/jira/browse/KAFKA-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16999546#comment-16999546
]
Chris Egerton commented on KAFKA-5756:
--------------------------------------
I believe a very similar issue may still be possible and that the
synchronization added in https://github.com/apache/kafka/pull/3702, while an
improvement, still doesn't prevent all possible errors caused by concurrent
calls to {{WorkerSourceTask::commitOffsets}} (which, as noted earlier in the
ticket, can from from both the periodic offset commit from the
{{SourceTaskOffsetCommitter}} class and the end-of-life offset commit from the
{{WorkerSourceTask}} itself).
The {{WorkerSourceTask}} class takes care to ensure that
{{OffsetStorageWriter::beginFlush}} isn't invoked concurrently, which was
implemented as part of https://github.com/apache/kafka/pull/3702. However,
there doesn't appear to be anything in place to prevent that method from being
invoked before a flush has completed (either via a call to
{{OffsetStorageWriter::cancelFlush}} or to
{{OffsetStorageWriter::doFlush::get}}). If this occurs, [an exception is
thrown|https://github.com/apache/kafka/blob/c6e25bb362899f4e6335ac5578b1cae31b7f2575/connect/runtime/src/main/java/org/apache/kafka/connect/storage/OffsetStorageWriter.java#L108-L112]
stating that "the framework should not allow this".
Reopening this issue until the above scenario has been addressed.
> Synchronization issue on flush
> ------------------------------
>
> Key: KAFKA-5756
> URL: https://issues.apache.org/jira/browse/KAFKA-5756
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.10.2.0
> Reporter: Oleg Kuznetsov
> Priority: Major
> Fix For: 0.11.0.1, 1.0.0
>
>
> Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()*
> method, whereas this collection can be accessed from 2 different threads:
> - *WorkerSourceTask.execute()*, finally block
> - *SourceTaskOffsetCommitter*, from periodic flush task
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
