slachiewicz opened a new pull request, #26: URL: https://github.com/apache/iceberg-verification/pull/26
Closes #13. Builds on #24, whose commit this branch carries until it merges; only the second commit is this change. `v3/row-lineage-parquet-data` has three snapshots. The first appends `a` (row IDs 0 to 2). The second appends `b` and `c` in one manifest (3 to 4, then 5), so a reader has to add the preceding file's `record_count`. The third is an overwrite that replaces `a` with `a2`, which stores row IDs 0 to 2 and sequence number 1 for the copied rows, a null sequence number for the updated row, and nulls for an added row that inherits `_row_id` 9 (`a2`'s `first_row_id` 6 plus position 3). A reader that ignores stored values returns 6 to 8 for the copied rows. `expected/scan.json` adds `_row_id` and `_last_updated_sequence_number` keyed by their reserved field IDs, and the README asks consumers to project them where `rows` carries those keys. The generator gains an `overwrite` commit whose rows may carry the two columns; the existing tables generate unchanged. The table is 52 KB over three commits; the overwrite writes a second manifest for the deleted entry. Checked at every snapshot: an Iceberg Java 1.11.0 generic scan projecting both columns returns `expected/scan.json`, and so does a read with no Iceberg library (plain Avro for manifest lists and manifests, DuckDB for Parquet). iceberg-go (1827440) resolves the same per-file `first_row_id` and data sequence numbers but cannot project `_row_id` yet. PyIceberg was not run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
