bk-mz opened a new issue, #9827:
URL: https://github.com/apache/iceberg/issues/9827
### Apache Iceberg version
1.4.3 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
Hey folks, when migrating from spark 3.4.1 to spark 3.5.0 we observe broken
behavior of MERGE INTO:
```
merge into database.table base_table
using batch
on
(base_table.data_load_ts >= TIMESTAMP '2024-02-19 15:00:00.0') and
(base_table.id = batch.id)
when MATCHED then UPDATE SET *
when not MATCHED then INSERT *
24/02/28 10:37:33 ERROR MicroBatchExecution: Query [id =
a43bffba-51a2-47b2-bf32-b978334ea6c9, runId =
7da1911b-8105-4b80-baa9-bf8ec4da9422] terminated with error
org.apache.spark.sql.catalyst.ExtendedAnalysisException:
[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column or function parameter with name
`base_table`.`data_load_ts` cannot be resolved. Did you mean one of the
following? [`base_table`.`data_load_ts`, `batch`.`data_load_ts`,
`base_table`.`date_sent`, `base_table`.`date_created`,
`base_table`.`date_updated`].; line 4 pos 5;
'MergeIntoTable (('base_table.data_load_ts >= 2024-02-19 15:00:00) AND
('base_table.id = 'batch.id)), [updatestaraction(None)],
[insertstaraction(None)]
```
The same works without problems on 3.4.1.
Please guide me if this is Apache Iceberg issue that you can help with.
Environment:
EMR 7.0.0
Spark 3.5.0
Iceberg 1.4.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
