Baunsgaard opened a new pull request, #3629: URL: https://github.com/apache/iceberg-rust/pull/3629
## Which issue does this PR close? - Closes #3471. ## What changes are included in this PR? `PromotePlan` now converts a file-side Arrow `FixedSizeList` to Iceberg `List`, the same way it already converts `LargeList`. parquet-rs can surface a 3-level Parquet `LIST` as `FixedSizeList` from the embedded Arrow schema; Iceberg has no fixed-size list type, so those scans used to fail with `cannot promote`. Null rows stay **null** with no children (pads from parquet-rs are dropped; Java `OptionReader` does the same). Arrow `cast` would keep the pads and fail `ListArray::try_new` when the element is required. This is reader encoding, not a new table type and not v4 vector (`apache/iceberg#18297`). `FixedSizeList` vs a non-list target is now `cannot promote` at plan time. That used to fall through to `Cast`, which unwrapped size-1 lists. A field-id table cannot pair list with primitive; a bad name mapping can. Not in this PR: #3467 (list-view hints), vector type. ## Are these changes tested? Unit tests in `record_batch_transformer.rs`: - `FixedSizeList<Int32>` -> `List<Int64>` with a null parent (`offsets` `[0, 2, 2]`) and child nulls - null parent + required element (does not fail `ListArray::try_new`) - all-non-null path through `RecordBatchTransformer` - `FixedSizeList` -> `Int32` rejected at plan time, including size 1 The bug is in `PromotePlan` after parquet-rs has already built a `FixedSizeListArray`, so the tests construct that array directly. A Parquet round-trip is omitted to keep the PR small. ## AI Disclosure AI (Cursor) was used to implement the change, write tests, and draft this description. I reviewed the spec (`list` vs `fixed(L)`), the `LargeList` path, and the transformer tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
