pvary commented on issue #10892: URL: https://github.com/apache/iceberg/issues/10892#issuecomment-2302445320
> @pvary I'd be happy to help if I can be helpful. Can you maybe expand on the 1st point? How do you envision it being implemented? Is it just as simple as when recovery is detected, remove the max checkpoint ID check and let Iceberg committer commit the data? We will still need to have `maxCommittedCheckpointId` in `initializeState` to prevent recommitting data to the Iceberg table on state restore, but in the `notifyCheckpointComplete` we need to have a different check. We need to correlate the current deployment and the last commit created by this deployment - we have to be aware of the lack of guarantees on the [CheckpointListener/notifyCheckpointComplete](https://nightlies.apache.org/flink/flink-docs-release-1.20/api/java//org/apache/flink/api/common/state/CheckpointListener.html#notifyCheckpointComplete-long-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
