codephage2020 commented on code in PR #9598:
URL: https://github.com/apache/arrow-rs/pull/9598#discussion_r3015283122
##########
parquet-variant-compute/src/variant_get.rs:
##########
@@ -86,15 +87,21 @@ pub(crate) fn follow_shredded_path_element<'a>(
return Ok(missing_path_step());
};
- let struct_array = field.as_struct_opt().ok_or_else(|| {
- // TODO: Should we blow up? Or just end the traversal and let
the normal
- // variant pathing code sort out the mess that it must anyway
be
- // prepared to handle?
- ArrowError::InvalidArgumentError(format!(
- "Expected Struct array while following path, got {}",
- field.data_type(),
- ))
- })?;
+ // The field might be a VariantArray (StructArray) if shredded,
+ // or it might be a primitive array. Only proceed if it's a
StructArray.
+ let Some(struct_array) = field.as_struct_opt() else {
+ // Field exists but is not a StructArray (VariantArray),
+ // which means it's not shredded further.
+ if !cast_options.safe {
+ return Err(ArrowError::CastError(format!(
+ "Expected Struct array while following path, got {}",
+ field.data_type(),
+ )));
+ }
+ // In safe mode, return NotShredded to let the caller
+ // handle it via value column.
+ return Ok(ShreddedPathStep::NotShredded);
Review Comment:
You're right. The proper behavior depends on whether a `value` column exists:
- If value column exists → `NotShredded` (data may be in the value column,
need per-row extraction)
- If value column is absent → `Missing` (the path truly doesn't exist,
return all-null)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
