BsoBird commented on code in PR #11787: URL: https://github.com/apache/iceberg/pull/11787#discussion_r1888149882
########## docs/docs/spark-writes.md: ########## @@ -101,6 +101,9 @@ Spark 3.5 added support for `WHEN NOT MATCHED BY SOURCE ... THEN ...` to update WHEN NOT MATCHED BY SOURCE THEN UPDATE SET status = 'invalid' ``` +!!! danger + Note: For copy on write table,Please Do Not Modify the Source Data Table During Execution.Due to the need for Spark to use the source table consecutively twice for computation,the relation which is created of source data must remain constant through the two different passes of the source data.If the source data query would return different results user will see odd behavior. Review Comment: > I'm trying to pierce the parts together, from Slack I understand that the source table is a Hive table, not Iceberg. Is that correct? @Fokko Yea,Sir. But in fact, other non-iceberg types of source tables could also cause this issue. Therefore, it would be best for us to do this: ``` before: merge into target_iceberg_cow_table t using (select * from dynamic_incr_table) s when xxxx; --data loss after: --for hive table(or external table)/jdbc view table,etc create table static_table as select * from dynamic_incr_table; --write a temp table(data not change) merge into target_iceberg_cow_table t using (select * from static_table ) s when xxxx; --for another datalake table merge into target_iceberg_cow_table t using (select * from dynamic_incr_table TIMESTAMP AS OF '2021-09-20 08:00:00') s when xxxx; --read a specific version ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
