sumedhsakdeo commented on PR #10336: URL: https://github.com/apache/iceberg/pull/10336#issuecomment-2226308996
Thank you so much @szehon-ho for your contribution to spark side. OPTIONS is indeed the right way to achieve this functionality. I was wondering if UPDATE and DELETE support is also in the pipeline, or someone else should pick it up. Our use-case is doing DELETES, using copy-on-write. While writing, it uses a 128MB split-size when we want the entire spark session to be configured with 512MB split-size. >> The problem with this is that it does lead to some ambiguity as to what table the config is applying to (many queries read from several tables, for example) I don't see ambiguity as spark session configs are meant to be visible across all reads that belong to the session. This is similar to how Iceberg solved it for `locality` flag. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
