raghav-reglobe opened a new pull request, #2678:
URL: https://github.com/apache/iceberg-rust/pull/2678
## Summary
Completes `RowDelta` with the **merge-on-read (MoR)** path — V3 Puffin
**deletion vectors** committed via a `content=Deletes` manifest — on top of
@wirybeaver's copy-on-write `RowDelta` foundation from #2203 (rebased onto
current `main`, authorship preserved). Together they give `RowDelta` both COW
(rewrite data files) and MoR (deletion vectors) — the basis for `MERGE INTO` /
`UPDATE` / `DELETE` on V3 tables.
#2203 has been stale and conflicting with `main`; per my note there, this
carries it forward. The rebase conflicts were trivial (additive `mod` / `use`
keep-both) plus one moved API (`Snapshot::load_manifest_list` →
`Table::manifest_list_reader`).
## MoR additions (on top of #2203)
- `DeleteVector::{to_puffin_blob, from_puffin_blob, write_to_puffin_file}` —
serialize/deserialize the `deletion-vector-v1` blob (magic `0xD1D33964`, 4-byte
BE length + portable 64-bit roaring bitmap + 4-byte BE CRC32; `snapshot-id` /
`sequence-number` = `-1`; `fields` = `[]`) and produce the `PositionDeletes`
`DataFile`.
- `RowDeltaAction::add_delete_files` MoR commit — writes a `content=Deletes`
manifest with `Operation::Delete` (replaces the previous `FeatureUnsupported`
stub).
- `delete_vector` made `pub` (with rustdoc).
## Validation
- **Byte-identical to Apache Iceberg-Java** — `to_puffin_blob` output equals
the Java-produced golden DV fixtures from `apache/iceberg` core test resources,
for 3 vectors: empty, `{1, 3, 5, 7, 9}`, and small+large spanning two roaring
containers (`crates/iceberg/testdata/puffin/*-position-index.bin`).
- **Cross-implementation parity** with `apache/iceberg-go`
(`puffin/dv_golden_test.go`, `TestSerializeDV*`).
- **Real data, cross-engine** — a deletion vector written by this code was
committed to a real V3 table via a REST catalog on S3 and read back correctly
by an independent engine (Apache Doris): 10 → 7 rows, exact positions deleted.
- 14 `delete_vector` + 7 `transaction::row_delta` tests pass; `cargo fmt` +
`cargo clippy` clean.
## Credit & follow-ups
- `RowDelta` copy-on-write foundation by @wirybeaver (#2203) — carried
forward here with authorship preserved.
- Out of scope (follow-ups): a higher-level `DVWriter` accumulator
(multi-blob-per-Puffin, dedup, partition capture) as in iceberg-go; reader-side
DV invariant enforcement (`snapshot-id`/`sequence-number` = -1 on read).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
