nssalian opened a new pull request, #28: URL: https://github.com/apache/iceberg-verification/pull/28
Closes https://github.com/apache/iceberg-verification/issues/19 The reference-tables have no table whose schema changes, so nothing currently checks that a reader fills a column added after data was written from its `initial-default`. This adds `v3/initial-default-parquet-data`. Snapshot 1 appends `a.parquet` with `id` and `data`. Then `quantity int` is added as required with `initial-default` 42, `note string` is added as optional with no default, and `quantity`'s `write-default` is changed to 10. Snapshot 2 appends `b.parquet` with all four columns. A scan at snapshot 2 returns the old rows with `quantity` 42 and `note` null, per the spec: "`initial-default` is used to populate the field's value for all records that were written before the field was added to the schema". The two defaults differ, so a reader that uses `write-default` for old rows fails. The table is generated by `GenerateTables.java` from the Input block in its README, which gains a `schema-updates` commit. The existing tables generate unchanged. Snapshot 1's expected rows assume a scan at an earlier snapshot uses that snapshot's schema, which is what every reader below does. This PR states that rule in the README's expected values and Procedure. Checked at both snapshots with a raw fastavro/pyarrow reader, PyIceberg 0.12.0, Iceberg Java 1.12.0, iceberg-go (d66ab48) and iceberg-rust 0.10.1 and 0.11.0. All match `expected/scan.json`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
