pvary commented on PR #9694: URL: https://github.com/apache/iceberg/pull/9694#issuecomment-1968835103
@mudit-97: If your job fails after the `snapshotState`, but before the `notifyCheckpointComplete`, and any of the other operators fail in the `snapshotState` then Flink could decide that the the checkpoint failed, and start from a previous checkpoint. In this case the data will be written again into the Iceberg table (we will have data duplication). https://en.wikipedia.org/wiki/Two-phase_commit_protocol -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
