wombatu-kun commented on PR #16438: URL: https://github.com/apache/iceberg/pull/16438#issuecomment-4506231418
@manuzhang quick update: after a few CI reruns with the observability commits in place, one of them caught the real failure. Stack trace pinned it down to a thread-safety bug, not the control-topic issue I originally suspected. `Coordinator.commit()` parallelizes per-table commits via `Tasks.foreach.executeWith(exec)`. Each parallel task calls `IcebergSinkConfig.tableConfig(...)`, which until now used `HashMap.computeIfAbsent` on a non-thread-safe map. On multi-table commits (`TestIntegrationDynamicTable.testIcebergSink`, `TestIntegrationMultiTable.testIcebergSink` — both write to 2 tables), the second thread's `computeIfAbsent` would see the first thread's insert mid-traversal and throw `ConcurrentModificationException`, killing the Coordinator after the first table committed but before the second. That's why the failure was always on the 2-table tests and always presented as "snapshot count 0 instead of 1" after the 30s Awaitility timeout. Fix is one line: switch `tableConfigMap` to `Maps.newConcurrentMap()`. Stack trace and CI run for reference: https://github.com/apache/iceberg/actions/runs/26213568037. This PR now contains: 1. `6c06dbeef` — isolate `iceberg.control.topic` per test (defense-in-depth; reduces how often the race fires under stale control-topic events, but doesn't eliminate it) 2. `232acd55` — the actual fix (`ConcurrentModificationException` in `IcebergSinkConfig.tableConfig`) Dropped the observability commits again as you originally suggested — #16504 carries them as a follow-up. Might also be worth retitling this PR to something like "Kafka Connect: Fix flaky multi-table integration tests by making IcebergSinkConfig.tableConfig thread-safe" if you agree the CME fix is now the primary change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
