Copilot commented on code in PR #3629:
URL: https://github.com/apache/iceberg-rust/pull/3629#discussion_r4474648114
##########
crates/iceberg/src/arrow/record_batch_transformer.rs:
##########
@@ -464,6 +483,30 @@ impl PromotePlan {
}
}
+/// parquet-rs pads each null FixedSizeList row with `size` null children;
drop them so a null
+/// list owns no child slots, as in a 3-level LIST and Java. Required elements
reject the pads.
+fn list_from_fixed_size_list(
+ fixed_size_list: &FixedSizeListArray,
+ element_field: &FieldRef,
+) -> Result<ArrayRef> {
+ let Some(nulls) = fixed_size_list.nulls().filter(|n| n.null_count() > 0)
else {
+ return Ok(cast(
+ fixed_size_list,
+ &DataType::List(Arc::clone(element_field)),
+ )?);
+ };
+ let size = fixed_size_list.value_length() as usize;
Review Comment:
Converting a valid `FixedSizeList` with more than `i32::MAX` retained
children panics in `OffsetBuffer::from_lengths` instead of returning an error.
The all-non-null `cast` branch has the same overflow problem; Boolean children
can reach this limit with about 256 MiB of value data. Check the retained child
count before either branch and return a conversion error if it exceeds the
`List` offset range.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
