RussellSpitzer commented on issue #17919: URL: https://github.com/apache/iceberg/issues/17919#issuecomment-5805188235
Implementation is going in as a series of small PRs following the separate-type design, starting with #18229. Only one is open at a time; the rest are staged locally and pushed as each merges. 1. #18229 — API: Add the file type 2. API, Core: Reserve and validate the derived field ID block 3. Core: Serialize the file type in schema JSON 4. API, Core, Data: Handle the file type in type switches and struct views 5. Core: Read and write file columns in Avro 6. Parquet: Map the file type to a group 7. ORC: Support the file type in schema conversion and visitors 8. Hive, Kafka Connect, AWS, Arrow: Render a file column as a struct Deferred follow-ups, filed separately so they don't block the series: - #18230 — Parquet FILE logical-type annotation (needs [parquet-java#3608](https://github.com/apache/parquet-java/pull/3608)) - #18231 — per-leaf statistics for file columns - #18232 — Spark and Flink support - #18233 — REST/OpenAPI and RCK coverage - #18234 — byte-resolution validation on write #17808 is closed in favor of this approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
