adriangb commented on PR #22698: URL: https://github.com/apache/datafusion/pull/22698#issuecomment-5567324576
Benchmark summary for head 12c9a05490 (no custom evaluator; learned order evaluated by `BinaryExpr` as a right-nested `AND`). Two runs on the same binary: flag off vs on ([trigger](https://github.com/apache/datafusion/pull/22698#issuecomment-5566318199)), and main vs PR with the flag off on both sides ([trigger](https://github.com/apache/datafusion/pull/22698#issuecomment-5566318428)). **Flag off → on** - tpch_sf10: Q6 **1.20x** and Q12 **1.21x** faster, everything else unchanged. Q12 was 1.45x with the earlier dedicated compact-once loop, so evaluating the learned order through `BinaryExpr` gives up about half of that win on a cheap 5-conjunct predicate. - clickbench: Q35 1.29x, Q36–Q42 1.10–1.32x faster; Q26/Q27 ~5–9% slower (tight stddev); net −0.6%. - tpcds_sf1: net **+1.8% slower**, with ~10 queries 5–14% slower (Q4, Q6, Q61, Q62, Q75, Q82 have tight stddev, the rest are noisy). These are cheap-comparison predicates: the settle guard only compares measured conjunct cost, not `BinaryExpr`'s per-level evaluation overhead, so it adopts reorders that buy nothing. Same effect as the k4 microbenchmark in the description. Fix: account for evaluator overhead in the guard (follow-up, or in this PR if preferred). **Main vs PR, flag off both sides** - tpcds and clickbench: neutral. - tpch: +2–3% total with Q9/Q13/Q18/Q22 6–13% slower. Q9 has a single-conjunct filter, which the flag-off path cannot touch, so this looks like binary layout / noise; a pinned A/A run on the PR head is queued to confirm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
