RussellSpitzer opened a new issue, #18230: URL: https://github.com/apache/iceberg/issues/18230
Follow-up to the `file` type work tracked in #17919. `TypeToMessageType.file()` currently writes a file column as a plain Parquet group with no logical type annotation, because `FileLogicalTypeAnnotation` does not exist in parquet 1.17.1. Iceberg readers resolve the nested fields by field ID so round trips work, but other readers see an ordinary group and cannot tell the column is a file. Two things are needed: - [apache/parquet-java#3608](https://github.com/apache/parquet-java/pull/3608) lands and is released. - Iceberg's `parquet` version moves past `1.17.1`. There is a TODO on the version in `gradle/libs.versions.toml` marking this. Then `TypeToMessageType.file()` should annotate the group, and `ParquetTypeVisitor` should dispatch `FILE`-annotated groups to a `file()` hook. Note that subclasses which rebuild the group (`RemoveIds`, `ApplyNameMapping`) will need to preserve the annotation, which the current `struct()` path drops. There is a TODO in `ParquetTypeVisitor` describing this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
