twuebi opened a new pull request, #1221: URL: https://github.com/apache/iceberg-go/pull/1221
Compaction replaces a data file carrying a deletion vector with new compacted files, dropping the old file's manifest entry, but leaves the DV's manifest entry pointing at a data file the table no longer references. ## Cause DV removal was keyed by path, but a DV entry is 1:1 with its referenced data file and one Puffin can hold DVs for several files. Compaction never collected DVs (they ride in `FileScanTask.DeletionVectorFiles`, not `DeleteFiles`), and a path-keyed removal would have dropped sibling DVs in the same Puffin too. ## Fix Key DV removal by referenced data file end to end. `CollectSafeDeletionVectors` gathers a group's DVs; `CompactionGroupResult.SafeDeletionVectors` carries them; both commit paths expunge them, dropping the rewritten file's DV while sibling DVs in the same Puffin survive. Use `RewriteFiles.ApplyResult`, not `Apply`, which can't carry them. ## Tests `TestRewriteDataFilesRemovesDeletionVectors` (no orphan after compaction) and `TestRewriteDataFilesPreservesSiblingDeletionVector` (shared Puffin: rewritten DV expunged, sibling survives). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
