slachiewicz opened a new pull request, #26:
URL: https://github.com/apache/iceberg-verification/pull/26

   Closes #13. Builds on #24, whose commit this branch carries until it merges; 
only the second commit is this change.
   
   `v3/row-lineage-parquet-data` has three snapshots. The first appends `a` 
(row IDs 0 to 2). The second appends `b` and `c` in one manifest (3 to 4, then 
5), so a reader has to add the preceding file's `record_count`. The third is an 
overwrite that replaces `a` with `a2`, which stores row IDs 0 to 2 and sequence 
number 1 for the copied rows, a null sequence number for the updated row, and 
nulls for an added row that inherits `_row_id` 9 (`a2`'s `first_row_id` 6 plus 
position 3). A reader that ignores stored values returns 6 to 8 for the copied 
rows.
   
   `expected/scan.json` adds `_row_id` and `_last_updated_sequence_number` 
keyed by their reserved field IDs, and the README asks consumers to project 
them where `rows` carries those keys. The generator gains an `overwrite` commit 
whose rows may carry the two columns; the existing tables generate unchanged.
   
   The table is 52 KB over three commits; the overwrite writes a second 
manifest for the deleted entry.
   
   Checked at every snapshot: an Iceberg Java 1.11.0 generic scan projecting 
both columns returns `expected/scan.json`, and so does a read with no Iceberg 
library (plain Avro for manifest lists and manifests, DuckDB for Parquet). 
iceberg-go (1827440) resolves the same per-file `first_row_id` and data 
sequence numbers but cannot project `_row_id` yet. PyIceberg was not run.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to