peterxcli commented on PR #23854: URL: https://github.com/apache/datafusion/pull/23854#issuecomment-5242287326
Gene’s review identified that the searched dynamic-filter `CASE` used `BinaryExpr` comparisons whose semantics do not exactly match the Range repartitioner’s `compare_rows` semantics. https://github.com/apache/datafusion/pull/23854/commits/8903354dec7671888e4ad0dff483aef920525bac fixes that issue by introducing `RangeExpr` and sharing the partition-ID implementation between Range repartitioning and dynamic-filter routing. While investigating signed zero, found a separate pre-existing issue: Range ordering can place two HashJoin-equal keys, such as `-0.0` and `+0.0`, into different partitions. This affects partitioned HashJoin even without dynamic filtering and is therefore outside the scope of the `RangeExpr` change. A general follow-up should make Range routing operate on canonicalized equi-join keys, likely using a shared canonicalizer and `RowConverter`, so equality always implies partition co-location. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
