ethan-tyler opened a new pull request, #1987:
URL: https://github.com/apache/iceberg-rust/pull/1987
## Summary
This PR adds delete file support to `SnapshotProducer`, enabling it to
accept and process both position delete files and equality delete files
alongside data files.
## Changes
- **New field**: `added_delete_files: Vec<DataFile>` in `SnapshotProducer`
- **New method**: `validate_added_delete_files()` for comprehensive
validation:
- Rejects delete files in format version 1
- Validates content types (PositionDeletes, EqualityDeletes)
- Requires `equality_ids` for equality delete files
- Validates partition spec compatibility
- **New method**: `write_delete_manifest()` to write delete manifests with
`ManifestContentType::Deletes`
- **Updated** `manifest_file()` to include delete manifests when present
- **Updated** `summary()` to populate delete file metrics in snapshot summary
- **Enhanced** `validate_duplicate_files()` to check both data and delete
files for duplicates
## Tests
Added 7 unit tests covering:
- Delete files rejected in V1 format
- Equality delete files require `equality_ids`
- Empty `equality_ids` rejected
- Data content type rejected for delete files
- Position delete validation succeeds
- Equality delete with ids succeeds
- Empty delete files returns appropriate error
## Motivation
This lays the groundwork for operations like `RowDelta` that need to
atomically commit both data files and delete files in a single snapshot. The
existing `SnapshotProducer` only supported data files.
## Notes
- The `validate_added_delete_files()` method is exposed as `pub(crate)` for
use by transaction actions that will use delete files
- Manifest-per-spec grouping for delete files is left as a follow-up (uses
default partition spec for now)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
