RussellSpitzer opened a new pull request, #18229: URL: https://github.com/apache/iceberg/pull/18229
First PR in a series implementing the Iceberg `file` type ([#17919](https://github.com/apache/iceberg/issues/17919), spec in [#17918](https://github.com/apache/iceberg/pull/17918)). This one adds the type itself and nothing else, so the remaining changes can be reviewed in small pieces. `Types.FileType` is its own `NestedType` with its own `TypeID.FILE`, rather than a subclass of `StructType`. Visitors that do not care about the distinction fall back to struct behavior: ```java public T file(Types.FileType file, List<T> fieldResults) { return struct(file.asStruct(), fieldResults); } ``` That fallback is what keeps this series small: most visitors, and all the engine integrations, keep compiling and working without a change. The six nested fields (`uri`, `offset`, `size`, `content_type`, `checksum`, `inline`) are derived from the enclosing field ID rather than stored, so a file column at ID *n* owns IDs *n+1* through *n+6*. This PR only teaches ID assignment to reserve that block so a file type survives a round trip. Validating that no other column claims an ID from the block is PR 2. A file column requires format version 4, gated through `Schema.MIN_FORMAT_VERSIONS`. ### Series outline Each of these is a separate follow-up PR; only this one is open right now. 1. **API: Add the file type** (this PR) 2. API, Core: Reserve and validate the derived field ID block 3. Core: Serialize the file type in schema JSON 4. API, Core, Data: Handle the file type in type switches and struct views 5. Core: Read and write file columns in Avro 6. Parquet: Map the file type to a group 7. ORC: Support the file type in schema conversion and visitors 8. Hive, Kafka Connect, AWS, Arrow: Render a file column as a struct Engine support (Spark, Flink) and the Parquet `FILE` logical-type annotation (which needs [parquet-java#3608](https://github.com/apache/parquet-java/pull/3608)) are deferred to separate issues. ### One open question for reviewers `TypeUtil.SchemaVisitor.file()` falls back to `struct()`, but `TypeUtil.CustomOrderSchemaVisitor.file()` throws `UnsupportedOperationException`. The inconsistency is deliberate for now — the custom-order visitors that need file handling override it explicitly in later PRs — but it may be worth making both consistent. --- **AI Disclosure** - Model: Claude Opus 4.5 - Platform/Tool: Cursor - Human Oversight: partially reviewed - Prompt Summary: Split an existing single-commit implementation of the Iceberg `file` type into a series of small, independently reviewable PRs, carve each from the integration tree, and verify each with the affected modules' test suites. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
