pvary opened a new pull request, #18485: URL: https://github.com/apache/iceberg/pull/18485
Draft of how we could add `int8`, `uint8`, and `float16` primitive types to the v4 table spec. Few notable things: - The types promote to `int`/`long` and `float`/`double`. - Bucket hashes match `int`/`long` (sign- vs zero-extension); `float16` is not bucketable, like `float`/`double`. - Truncate returns `int` type for `int8` as truncated value could be out of range (-128) and type promotion would change the behaviour. Same for `unit8` for now for symmetry - we could reconsider it Avro stores the wider `int`/`float` with Iceberg-convention logical types, Parquet uses `INT(8, signed)` on `int32` and `FLOAT16` ORC uses `byte`, `short`, and `float` with type attributes. Adding `float16` requires Parquet format version 2.10 --- **AI Disclosure** - Model: [unknown - human to fill in] - Platform/Tool: GitHub Copilot (JetBrains IDE agent) - Human Oversight: reviewed, iterated - Prompt Summary: Add int8, uint8, and float16 column types to the Iceberg table and expressions specs, including type promotion, transforms, serialization, and Avro/Parquet/ORC mappings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
