rgbuilds opened a new pull request, #26996: URL: https://github.com/apache/datafusion/pull/26996
## Which issue does this PR close? - Part of #25616. ## Rationale for this change Join cardinality estimation accounts for equality join keys but not residual join predicates. This can overestimate output rows when a residual comparison filters candidate pairs. ## What changes are included in this PR? - Add a reusable interval comparison-selectivity helper for `<`, `<=`, `>`, and `>=`. - Pass residual filters into join statistics estimation. - Map the filter’s compact columns to input column statistics. - Apply residual selectivity to inner joins with direct column-to-column comparisons. - Preserve the existing estimate for unsupported expressions, casts, types, statistics, and non-inner joins. Cast handling is intentionally excluded from this first PR, following the agreed scope. Therefore, the original TPC-H cast-containing predicate still falls back to the existing estimate. ## What is the testing strategy for this PR? Added focused tests covering: - Comparison probabilities, including `0.49` for `[1, 50] < [1, 50]`. - Disjoint and single-value intervals. - Unsupported inputs and fallback behavior. - Inner and non-inner join estimates. - `HashJoinExec` statistics integration. - The remaining TPC-H cast limitation. Formatting, full workspace clippy, and the extended workspace test suite pass. ## Are there any user-facing changes? Join row-count estimates improve for supported residual comparisons. There are no breaking API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
