RussellSpitzer commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1767613662
########## format/spec.md: ########## @@ -222,11 +228,31 @@ Schemas may be evolved by type promotion or adding, deleting, renaming, or reord Evolution applies changes to the table's current schema to produce a new schema that is identified by a unique schema ID, is added to the table's list of schemas, and is set as the table's current schema. -Valid type promotions are: +Valid primitive type promotions are: + +| Primitive type | v1, v2 valid type promotions | v3+ valid type promotions | Requirements | +|------------------|------------------------------|------------------------------|--------------| +| `unknown` | | _any type_ | | +| `int` | `long` | `long` | | +| `date` | | `timestamp`, `timestamp_ns` | Promotion to `timestamptz` or `timestamptz_ns` is **not** allowed | +| `float` | `double` | `double` | | +| `decimal(P, S)` | `decimal(P', S)` if `P' > P` | `decimal(P', S)` if `P' > P` | Widen precision only | -* `int` to `long` -* `float` to `double` -* `decimal(P, S)` to `decimal(P', S)` if `P' > P` -- widen the precision of decimal types. +Iceberg's Avro manifest format does not store the type of lower and upper bounds, and type promotion does not rewrite existing bounds. For example, when a `float` is promoted to `double`, existing data file bounds are encoded as 4 little-endian bytes rather than 8 little-endian bytes for `double`. To correctly decode the value, the original type at the time the file was written must be inferred according to the following table: Review Comment: We discussed this a little, but I want to double check. We aren't allowing any promotion to variant here correct? It would basically be impossible to reverse engineer the original write time metric so I assume we are giving up on that. It could be possible in the future if we change the metric layout. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
