limadog9 opened a new pull request, #25835: URL: https://github.com/apache/datafusion/pull/25835
## Which issue does this PR close? Closes #25614. ## Rationale for this change String equality, LIKE, and NOT LIKE currently receive the same fallback estimate. For a column with 1,000 rows, 200 NULLs, and 20 distinct non-null values, equality should estimate 40 matching rows and inequality 760, instead of estimating 200 for both. ## What changes are included in this PR? - Estimate direct string equality/inequality using the distinct count when available, under the existing uniform-distribution assumption. - Classify LIKE patterns as literal, all-percent, or wildcard patterns. Wildcard estimates use the configured fallback, halved for each anchored end. Recognize escaped wildcards and Arrow's literal trailing-backslash behavior. - Complement negated estimates over non-null rows and propagate zero null counts for LIKE operands. Literal ILIKE retains a heuristic because case folding can merge distinct values. - Retain existing estimates for compound or otherwise unsupported expressions. - Remove an unnecessary semicolon in the Windows rounding implementation so the required Clippy check can run without that pre-existing warning. ## What is the testing strategy for this PR? Public SQL planning tests in `datafusion/core/tests/custom_sources_cases/statistics.rs` inject known statistics through the existing table-provider fixture. They cover Utf8, LargeUtf8, Utf8View, equality/inequality, pattern forms, escaped characters, ILIKE, unknown distinct counts, and configured fallback selectivity. Direct physical-plan tests cover patterns eliminated by SQL simplification, NULL/empty inputs, and column-valued patterns. Validation: - 50 filter tests passed. - 3 new public SQL regression tests passed. - `cargo fmt --all` passed. - Ablation passed: all 3 new SQL tests fail with the original `main` filter implementation, with the expected row-count assertion failures. - TOML/Markdown formatting, license headers, typos, workflow checks, and ASF YAML checks passed. Full workspace validation remains pending. ## Are there any user-facing changes? String filter row estimates change. No public API changes. Wildcard estimates remain heuristics in the absence of string histograms. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
