Xuanwo opened a new pull request, #25728: URL: https://github.com/apache/datafusion/pull/25728
## Which issue does this PR close? - Part of #318. - Follow-up to #25574. This is stacked on #25574 and will include that PR's commit until it merges. ## Rationale for this change #25574 mirrors a selective `left_key = literal` filter to the right side of an ASOF join. The same equality guarantee applies to any deterministic predicate that references only direct left equality-key columns, including `IN`, ranges, and `OR` expressions. Mirroring those predicates avoids sorting and broadcasting right-side groups that cannot match a surviving left row. ## Performance Q10 has 50M rows on each side across 500K equality groups. `WHERE l.key IN (42, 43)` retains two groups and produces 200 rows with both revisions. On an Apple M4 Max, using independent release builds and two 7-iteration runs per revision, the median elapsed time across all 14 iterations was: | Revision | Median elapsed | | --- | ---: | | #25574 without this follow-up (`c7522fa37`) | 974.8 ms | | This change (`5182481ce`) | 78.0 ms | This selective compound-key workload is about **12.5x faster** (92.0% lower elapsed time). The benchmark file and command were the same for both binaries: ```shell target/release/benchmark_runner asof_join --query 10 --iterations 7 ``` ## What changes are included in this PR? - Build an ASOF-specific column mapping for direct equality keys. - Mirror a pushed deterministic predicate only when every referenced column has a right-key mapping. - Support compound `IN`, range, and `OR` predicates while leaving mixed key/non-key predicates entirely off the right input. - Add plan coverage and ASOF benchmark Q10. Coerced and expression-based equality keys remain excluded because they cannot be replaced column-for-column. ## What is the testing strategy for this PR? The new `asof_join.slt` cases verify compound predicate mirroring and the all-or-nothing behavior for a mixed key/non-key `OR` predicate. Q10 covers the user-visible selective workload. Also run: - `cargo fmt --all -- --check` - `cargo clippy --all-targets --all-features -- -D warnings` - The extended workspace test suite, including all 524 sqllogictest files ## Are there any user-facing changes? No API or result changes. More eligible ASOF joins can sort and retain fewer right-side rows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
