Copilot commented on code in PR #24359:
URL: https://github.com/apache/datafusion/pull/24359#discussion_r3781825004
##########
datafusion/datasource-parquet/src/opener/mod.rs:
##########
@@ -1464,6 +1464,14 @@ impl RowGroupsPrunedParquetOpen {
};
let prepared_access_plan = prepare_access_plan(access_plan)?;
+ // #24355: a page-index row selection is carried by the decoder as
one
+ // flat selection over the concatenation of the remaining row
groups.
+ // The runtime pruner's `into_builder().with_row_groups(...)`
rebuild
+ // drops row groups without slicing that selection to match, so
record
+ // whether a selection is present and disable runtime pruning below
Review Comment:
The comment says this is specifically a *page-index* row selection, but
`prepared_access_plan.row_selection` can be produced by multiple pruning
mechanisms (stats/page index/bloom/etc.). Consider wording this as “any
row_selection” (or “row_selection, e.g. from page-index”) to match what the
code is actually checking.
This issue also appears on line 1515 of the same file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
