viirya commented on code in PR #2635:
URL: https://github.com/apache/iceberg-rust/pull/2635#discussion_r3414347204
##########
crates/iceberg/src/arrow/value.rs:
##########
@@ -628,23 +627,25 @@ pub(crate) fn create_primitive_array_single_element(
data_type: &DataType,
prim_lit: &Option<PrimitiveLiteral>,
) -> Result<ArrayRef> {
+ // With no value, the single element is NULL. `new_null_array` supports
every
+ // Arrow type, including nested ones (list/map/struct), which matters for
+ // columns added by schema evolution after a data file was written (#2618).
+ if prim_lit.is_none() {
Review Comment:
Thanks @advancedxy — agreed on both counts, and thanks for flagging the V3
angle.
You're right that `create_primitive_*` is now a misnomer since these
functions materialize list/map/struct NULLs too. I dug into the V3
default-value direction a bit, and it's actually a slightly larger refactor
than just the helper signatures: the value currently threaded into them is
`Option<PrimitiveLiteral>` (via `ColumnSource::Add`), and
`generate_transform_operations` deliberately drops any non-primitive
`initial_default` today (`if let Literal::Primitive(prim) = lit { .. } else {
None }` in `record_batch_transformer.rs`). So supporting a struct/nested
default would mean widening `ColumnSource::Add.value` to `Option<Literal>` and
the transformer's default-extraction alongside the rename to
`create_array_single_element`.
Given that, I'd prefer to keep this PR as the focused wraparound/nested-NULL
fix and do the rename + `Literal` widening + V3 default support together in the
follow-up, so the type change lands in one coherent step rather than renaming
now and re-touching the signature later. Happy to open the follow-up issue/PR
for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
