jun-su-lee commented on PR #17593: URL: https://github.com/apache/iceberg/pull/17593#issuecomment-5408538800
Hi @bryanck @danielcweeks, when you have a moment could you take a look at this PR? It fixes idle Kafka Connect sink tasks stalling every commit round: a task that holds partitions but receives no records never had a worker started (the worker's lifetime was tied to record arrival rather than to the assignment), so it never answered the coordinator's commit requests and each round fell back to a partial commit after the timeout. The fix starts the worker whenever the task owns an assignment (`ownsPartitions()`), driven from `save` where the control-topic polling happens. `TestCommitterImplWorkerLifecycle` adds coverage for the idle-task and cooperative-rebalance cases, verified by mutation. Since you two authored / merged most of the commit-coordination code here, your review would be much appreciated. Happy to adjust anything. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
