bryanck opened a new pull request, #10814: URL: https://github.com/apache/iceberg/pull/10814
This PR aims to improve the handling of coordinator shutdown during sink rebalances, to better ensure only one coordinator is active at a given time. The sink was designed with the assumption that only one coordinator and its tasks are active, to achieve exactly once semantics. When a rebalance is triggered and `close()` is called on the sink task, here we immediately flag the coordinator as terminated, shut down the commit executor, and await termination from the main sink task thread (instead of from the coordinator thread). Any new table commits attempted by this coordinator will check the terminated flag and fail if set. This doesn't solve all edge cases but should be an improvement to fencing out zombie coordinators. This PR also updates some exceptions thrown from `RuntimeException` to `ConnectException` to be a bit more specific. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
