adriangb commented on PR #25677: URL: https://github.com/apache/datafusion/pull/25677#issuecomment-5808006281
## Benchmark summary Times are the minimum of 5 iterations from the bot tables. Change = PR / base - 1. Only queries that changed by more than 5% are listed. | Suite | Config | Result | Total base | Total PR | Change | |---|---|---|---|---|---| | TPC-H SF1 | default | [link](https://github.com/apache/datafusion/pull/25677#issuecomment-5807904310) | 738.3 ms | 740.4 ms | +0.3% | | TPC-DS SF1 | default | [run 1](https://github.com/apache/datafusion/pull/25677#issuecomment-5807922464) | 9206.8 ms | 9131.3 ms | -0.8% | | TPC-DS SF1 | default | [run 2](https://github.com/apache/datafusion/pull/25677#issuecomment-5807998017) | 9043.1 ms | 9175.7 ms | +1.5% | | TPC-H SF1 | `pushdown_filters=true` | [link](https://github.com/apache/datafusion/pull/25677#issuecomment-5807909541) | 858.7 ms | 828.2 ms | **-3.6%** | | TPC-H SF10 | `pushdown_filters=true` | [link](https://github.com/apache/datafusion/pull/25677#issuecomment-5807928205) | 7867.5 ms | 7601.8 ms | **-3.4%** | | TPC-DS SF1 | `pushdown_filters=true` | [link](https://github.com/apache/datafusion/pull/25677#issuecomment-5807920778) | 7818.7 ms | 7408.3 ms | **-5.2%** | ### `pushdown_filters=true` | Query | Base | PR | Change | |---|---|---|---| | TPC-H SF1 Q17 | 71.8 ms | 44.9 ms | **-37.5%** | | TPC-H SF1 Q18 | 75.2 ms | 65.7 ms | **-12.7%** | | TPC-H SF1 Q20 | 30.6 ms | 32.5 ms | +6.2% | | TPC-H SF1 Q22 | 16.5 ms | 17.5 ms | +6.0% | | TPC-H SF10 Q18 | 650.8 ms | 408.6 ms | **-37.2%** | | TPC-H SF10 Q11 | 71.5 ms | 65.7 ms | -8.2% (noise, means equal) | | TPC-DS Q17 | 115.6 ms | 73.4 ms | **-36.5%** | | TPC-DS Q18 | 161.8 ms | 115.1 ms | **-28.8%** | | TPC-DS Q25 | 116.7 ms | 73.9 ms | **-36.7%** | | TPC-DS Q29 | 147.2 ms | 86.8 ms | **-41.0%** | | TPC-DS Q64 | 508.1 ms | 336.0 ms | **-33.9%** | | TPC-DS Q65 | 158.7 ms | 147.1 ms | -7.3% | | TPC-DS Q95 | 91.8 ms | 56.8 ms | **-38.1%** | No TPC-DS query is slower by more than 5% with pushdown on. ### Default config (`pushdown_filters=false`) TPC-H SF1: all queries within ±3%. TPC-DS was run twice because run 1 moved many queries in both directions. The table lists only the queries that moved more than 4.5% in the same direction in both runs: | Query | Run 1 base → PR | Run 2 base → PR | Change (run 1 / run 2) | |---|---|---|---| | Q24 | 186.2 → 215.0 ms | 190.6 → 206.3 ms | +15.5% / +8.2% | | Q27 | 5.8 → 6.8 ms | 5.9 → 6.4 ms | +17.1% / +8.1% | | Q30 | 33.7 → 35.8 ms | 31.9 → 34.4 ms | +6.4% / +7.8% | | Q83 | 33.5 → 36.4 ms | 33.0 → 35.3 ms | +8.7% / +6.9% | | Q84 | 29.5 → 31.2 ms | 29.6 → 31.0 ms | +5.8% / +4.6% | | Q85 | 103.3 → 109.0 ms | 103.4 → 108.1 ms | +5.5% / +4.6% | | Q6 | 39.4 → 34.3 ms | 37.5 → 35.1 ms | -13.2% / -6.3% | | Q46 | 12.9 → 11.6 ms | 12.7 → 11.6 ms | -10.4% / -8.6% | | Q53 | 30.7 → 28.6 ms | 30.8 → 28.9 ms | -6.8% / -6.1% | These queries do not move with pushdown on (all within ±4%). Treat them as possible small effects, not confirmed. **Conclusion:** with `pushdown_filters=true`, 9 query runs (TPC-H Q17 and Q18, TPC-DS Q17, Q18, Q25, Q29, Q64 and Q95) are 12-41% faster and the suite totals are 3-5% lower. With the default config the totals do not change (within ±1.5%). Default TPC-DS shows small, repeated slowdowns (5-15%) in Q24, Q30, Q83 and Q85 that could be worth a look. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
