kosiew commented on code in PR #24699:
URL: https://github.com/apache/datafusion/pull/24699#discussion_r3870189952
##########
datafusion/common/src/nested_struct.rs:
##########
@@ -215,6 +275,17 @@ pub fn cast_column(
target_value_type,
cast_options,
),
+ (
+ DataType::Union(source_fields, source_mode),
+ DataType::Union(target_fields, target_mode),
+ ) => cast_union_column(
+ source_col,
+ source_fields,
+ source_mode,
+ target_fields,
+ target_mode,
+ cast_options,
+ ),
_ => Ok(cast_with_options(source_col, target_type, cast_options)?),
Review Comment:
Could we add a focused `MemoryStream` regression for a contained `Map<...,
Struct>` where the runtime nested field is non-nullable and the declared nested
field is nullable? My understanding is that this should already work because
shapes accepted by `Schema::contains` retain the same Map entry/key/value
structure, and Arrow's generic Map cast should recursively cast the value
Struct and rebuild it with the target fields. A regression here would confirm
that assumption for this branch.
```
If it fails, we should either backport the minimal Map prerequisite from
https://github.com/apache/datafusion/pull/23914 or explicitly constrain
the branch-55 adapter to shapes this branch can safely normalize.
```
as suggested by @patrickswedish
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
