suisenkotoba commented on issue #9960: URL: https://github.com/apache/iceberg/issues/9960#issuecomment-2812629678
@nastra my pipeline is using dataproc, and it is scheduled to run daily. It will create new dataproc cluster every run, after jobs completed the cluster will be destroyed. So yes, it always use fresh spark installation. > https://github.com/apache/spark/pull/41028 added UPDATE handling to Spark 3.5. At the same time, UPDATE handling was removed from Iceberg's Spark 3.5 runtime module (because it's now completely handled by Spark). Sorry I don't quite get this. Does the UPDATE available on spark 3.5 or not? other than using dml update like this: `spark.sql("UPDATE {table} SET ... WHERE ...")` is there another way to update the table and incorporate it into pyspark codes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
