Beutlin opened a new pull request, #18123: URL: https://github.com/apache/iceberg/pull/18123
IcebergCommitter#commit unconditionally looked up flink.max-committed-checkpoint-id for the incoming committable's jobId-operatorId pair. On a stateless restart the checkpoint counter starts over from 1 while the table still carries the previous run's higher watermark for that same pair, so every new committable was marked as already committed and silently dropped without producing a snapshot. Gate the lookup on whether the committer actually restored state, mirroring the isRestored() check already used by the legacy IcebergFilesCommitter. On a stateless start there is no restored Flink state and therefore no possibility of a redelivered committable, so every committable can be committed unconditionally. This change was drafted with AI assistance and verified by running the existing and new IcebergCommitter tests, plus a manual regression check confirming the new tests fail without the fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
