mxm opened a new pull request, #16874: URL: https://github.com/apache/iceberg/pull/16874
Adds EqualityConvertCommitter, a two input operator for the equality delete conversion task. It buffers the DVWriteResults from the parallel EqualityConvertDVWriter instances and the EqualityConvertPlan from the planner, then commits once both the plan and its done-timestamp watermark have arrived. The committer adds both new staging data files and the writer's merged DVs, removes the superseded DVs, and carries over the remaining staging deletes. It validates against the planner's main snapshot, so external changes on the target branch fail the commit. The commit summary records the processed staging snapshot id, which the planner reads on restart to skip already-committed snapshots, to ensure idempotency. The committer is responsible for sending Trigger records downstream to the TaskResultAggregator of the surrounding table maintenance framework. It emits one after every cycle, including no-op and error, so the task always completes. On an upstream abort or a commit failure it deletes the DVs written this cycle so the failure doesn't leak Puffin files. This commit is factored out of #15996. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
