Johan Lasperas created SPARK-56054:
--------------------------------------
Summary: Fix aliased nested fields ignored for schema evolution in
MERGE
Key: SPARK-56054
URL: https://issues.apache.org/jira/browse/SPARK-56054
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.2.0
Reporter: Johan Lasperas
Small bug in the [initial implementation of schema evolution in
MERGE](https://github.com/apache/spark/pull/51698): when a struct in the source
is used aliased in a direct assignment clause in MERGE, it's fields aren't
considered for schema evolution.
Example:
```
source.mergeInto("target", condition)
.whenMatched()
update(Map("info" -> col("source.info").as("info")))
.withSchemaEvolution()
.merge()
```
where `info` is a struct that contains an extra field in the source compared to
the target. The extra field is ignored during schema evolution and isn't added
to the target table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
