tmnd1991 commented on PR #9233: URL: https://github.com/apache/iceberg/pull/9233#issuecomment-1847976194
Finally I got a reproducer inside the codebase, you can find it at `TestSPJWithBucketing`. Spark 3.4 (same as my app) with the condition on the partitions will actually prune the unaffected partitions, while 3.5 will not. Anyway the more I work on this, the more I think the issue should be solved directly on the Scan, not by adding conditions manually. All info should be available to Spark beforehand, am I right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
