zhuqi-lucas commented on code in PR #21674:
URL: https://github.com/apache/datafusion/pull/21674#discussion_r3093277104
##########
benchmarks/src/sort_pushdown.rs:
##########
@@ -159,7 +159,11 @@ impl RunOpt {
async fn benchmark_query(&self, query_id: usize) ->
Result<Vec<QueryResult>> {
let sql = self.load_query(query_id)?;
- let config = self.common.config()?;
+ let mut config = self.common.config()?;
+ // Enable filter pushdown so TopK's dynamic filter is pushed to the
+ // parquet reader for late materialization — only sort-column rows
+ // pass the filter, non-sort columns are skipped for filtered rows.
+ config.options_mut().execution.parquet.pushdown_filters = true;
Review Comment:
Good catch! Fixed — reverted the sort_pushdown.rs change and moved
pushdown_filters to an env var only for the inexact benchmark run. Other
sort_pushdown benchmarks are unaffected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
