yadavay-amzn opened a new pull request, #16237: URL: https://github.com/apache/iceberg/pull/16237
Fixes #15878. ## Problem The Kafka Connect `Coordinator` previously caught `Exception` around `doCommit()` and only logged a warning, so when a commit failed (e.g., a `CommitFailedException` from Glue detecting a concurrent table update), the connector stayed `RUNNING` while silently dropping the data that was in flight. ## Fix Remove the catch-all around `doCommit()` and instead log at `ERROR` level with the task id and commit id before rethrowing. `CoordinatorThread.run()` already terminates the thread on uncaught exceptions, which transitions the Kafka Connect task to `FAILED` — so failures are now surfaced rather than dropped. The `finally` block that calls `commitState.endCurrentCommit()` is preserved so per-commit state is cleaned up regardless of the outcome. ## Testing - Added `testCommitFailedExceptionPropagates` which mocks a catalog-side `CommitFailedException` on `AppendFiles.commit()` and asserts it propagates out of `Coordinator.process()`. Without the fix, this test fails because the exception is swallowed. - Updated two existing tests (`testCoordinatorWithBadDataFile` and `testCoordinatorCommittedOffsetValidation`) that previously relied on silent-swallow behaviour; they now assert the specific exception propagates (`IllegalArgumentException` for bad partition spec, `ValidationException` for stale offsets). - Full `TestCoordinator` suite passes locally (8/8). - `spotlessApply` passes. ## Generative AI Generated-by: Claude Opus 4.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
