mdashti opened a new pull request, #23106: URL: https://github.com/apache/datafusion/pull/23106
> [!NOTE] > Stacked on #23104, which adds the shared probe-NULL helper this builds on. That one should land first; until it does, this PR's diff includes its two commits. ## Which issue does this close? Re-enables the dynamic filter that #22965 disabled (#22964), with the proper null-equal semantics. ## Rationale for this change #22965 disabled hash-join dynamic filter pushdown for null-equal joins: the build-side bounds and membership predicates evaluate to NULL for a probe-side NULL key, so they prune rows that should null-match a build-side NULL. Its description already named the better fix, "generate a predicate with `OR IS NULL`". #23104 does that for null-aware anti joins; this re-enables the null-equal case the same way. ## What changes are included in this PR? - Revert the null-equal `return false` in `allow_join_dynamic_filter_pushdown`. - Generalize the shared probe-NULL helper to cover both null-aware (single-key) and null-equal (multi-key) joins: OR `key IS NULL` for every nullable probe key. A NOT NULL key never widens the filter, so an all-NOT-NULL join keeps full selectivity. ## Are these changes tested? Yes. #22965's SLT now asserts the filter is back on the probe with the result unchanged, plus a multi-key null-equal case. The unit test flips to assert pushdown is allowed. ## Are there any user-facing changes? Null-equal joins regain dynamic filter pushdown, so they prune the probe scan again while returning correct results. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
