azwanzuharimi commented on PR #3509: URL: https://github.com/apache/iceberg-python/pull/3509#issuecomment-5886555034
Thank you for this PR. I tested it as part of #4027. Prefix grouping fixes the low cardinality shape: one unique column plus one column with about 10 values, 20,000 rows, pass in 0.3 s. When both key columns are unique, it still emits one disjunct per key and crashes at 20,000 rows (exit 138 on macOS, pyarrow 25.0.1). The insert and overwrite paths also still build the exact Or. #4027 takes a different route: one `In` per column plus an exact anti join in Arrow. The full result table is in that PR. Happy to fold the tests into this PR if the maintainers prefer it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
