pepijnve opened a new pull request, #25023:
URL: https://github.com/apache/datafusion/pull/25023
## Which issue does this PR close?
- Closes #25022.
## Rationale for this change
`nested_struct::validate_struct_compatibility` checks for at least one
common field between the source and target field sets. This does not take into
account that the source field set might be empty. This can cause issue during
query planning.
For instance, when querying an Iceberg table's manifest with no partitioning
information the following error is produced:
```
Execution error: Cannot cast column 'data_file' from 'Struct("content":
non-null Int32, "file_path": non-null Utf8, "file_format": non-null Utf8,
"partition": non-null Struct(), metadata: {"avro.name": "r102"},
"record_count": non-null Int64, "file_size_in_bytes": non-null Int64,
"column_sizes": List(non-null Struct("key": non-null Int32, "value": non-null
Int64), metadata: {"avro.name": "k117_v118"}), "value_counts": List(non-null
Struct("key": non-null Int32, "value": non-null Int64), metadata: {"avro.name":
"k119_v120"}), "null_value_counts": List(non-null Struct("key": non-null Int32,
"value": non-null Int64), metadata: {"avro.name": "k121_v122"}),
"nan_value_counts": List(non-null Struct("key": non-null Int32, "value":
non-null Int64), metadata: {"avro.name": "k138_v139"}), "lower_bounds":
List(non-null Struct("key": non-null Int32, "value": non-null Binary),
metadata: {"avro.name": "k126_v127"}), "upper_bounds": List(non-null
Struct("key": non-null Int32, "value": non-null B
inary), metadata: {"avro.name": "k129_v130"}), "key_metadata": Binary,
"split_offsets": List(non-null Int64), metadata: {"element-id": "133"},
"equality_ids": List(non-null Int32), metadata: {"element-id": "136"},
"sort_order_id": Int32, "referenced_data_file": Utf8)' (physical data type) to
'Struct("content": non-null Int32, "file_path": non-null Utf8, "file_format":
non-null Utf8, "partition": non-null Struct(), "record_count": non-null Int64,
"file_size_in_bytes": non-null Int64, "column_sizes": List(non-null
Struct("key": non-null Int32, "value": non-null Int64)), "value_counts":
List(non-null Struct("key": non-null Int32, "value": non-null Int64)),
"null_value_counts": List(non-null Struct("key": non-null Int32, "value":
non-null Int64)), "nan_value_counts": List(non-null Struct("key": non-null
Int32, "value": non-null Int64)), "lower_bounds": List(non-null Struct("key":
non-null Int32, "value": non-null Binary)), "upper_bounds": List(non-null
Struct("key": non-null Int32, "val
ue": non-null Binary)), "key_metadata": Binary, "split_offsets": List(non-null
Int64), "equality_ids": List(non-null Int32), "sort_order_id": Int32,
"referenced_data_file": Utf8)' (logical data type): Error during planning:
Cannot cast struct with 0 fields to 0 fields because there is no field name
overlap
```
## What changes are included in this PR?
- Adds an early success return when `source_fields.is_empty()`
## What is the testing strategy for this PR?
- Added unit tests
- Verified SLTs still pass
## Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
