laskoviymishka opened a new pull request, #1298: URL: https://github.com/apache/iceberg-go/pull/1298
Two related fixes for optimistic-concurrency validation of position deletes. 1. validateDataFilesExist counted a referenced data file as present whenever any manifest entry for its path existed on the branch head, because it iterated head.dataFiles (discardDeleted=false). A concurrent compaction or overwrite that rewrote the file leaves a DELETED tombstone, so the check passed and the orphaned position delete (its rows had moved to a new file) was committed silently. Iterate data-manifest entries and treat a DELETED entry as absent — only a live ADDED/EXISTING entry satisfies existence, mirroring Java's MergingSnapshotProducer. This corrects the check for every caller, including RowDelta. 2. performMergeOnReadDeletion never ran that check at all: it builds an overwriteFiles producer whose only validator checks partition-filter overlap, and appended its position deletes without registering validateDataFilesExist (wired only into RowDelta.validate). So a merge-on-read delete whose referenced data file was concurrently removed went undetected. writePositionDeletesForFiles now returns the referenced data-file paths of the position deletes it appends, and performMergeOnReadDeletion registers a validateDataFilesExist validator over them, mirroring RowDelta.validate: unconditional (no isolation gating) and additive to the producer's partition-filter validator. A position delete that does not record its referenced data file is skipped, matching RowDelta and Java. Tests: TestValidateDataFilesExist_TreatsConcurrentlyRemovedFileAsMissing pins the status-aware check (a compacted-away file is missing; a live file is present); TestMergeOnReadDeleteConflict_ReferencedDataFileRemoved drives a merge-on-read delete whose referenced file a concurrent compaction removes and asserts the commit fails with ErrDataFilesMissing; the no-conflict control still commits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
