mxm opened a new pull request, #14182: URL: https://github.com/apache/iceberg/pull/14182
Previously, the DynamicCommitter could commit duplicate WriteResults when recovering from failures, leading to incorrect data in tables. This change introduces tracking of the maximum committed WriteResult index per checkpoint to ensure idempotent behavior during recovery scenarios. Key changes: - Added MAX_WRITE_RESULT_INDEX snapshot property to track committed WriteResults - Modified commit logic to skip already committed WriteResults within a checkpoint - Optimized atomic commits by batching append-only WriteResults into single transactions - Updated tests to verify idempotent behavior with simulated failures -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
