kinolaev opened a new pull request, #18509:
URL: https://github.com/apache/iceberg/pull/18509
On v3 tables, `RowDelta` always validates added DVs. Flink committers don't
set a starting snapshot, so the validation walks the whole branch history and
fails with `Cannot determine history between starting snapshot null and the
last known ancestor` once the oldest snapshot has been expired. The committer
then can't commit, including on restore. `IcebergCommitter`,
`IcebergFilesCommitter` and `DynamicCommitter` now refresh the table and
validate from the current branch head on v3+ tables, which matches what Spark
sets. v2 commits are unchanged.
Test plan: for each committer, a v3 test commits two checkpoints, expires
the oldest snapshot, then commits a data file with a DV, and asserts the commit
lands. Each test fails with the error above without the fix.
---
**AI Disclosure**
- Model: Claude Opus 5.5
- Platform/Tool: Claude Code
- Human Oversight: fully reviewed
- Prompt Summary: Diagnose a Flink commit failure from a production log
("Cannot determine history ... starting snapshot null") and fix it in the Flink
sink committers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
