rgbuilds commented on code in PR #25819:
URL: https://github.com/apache/datafusion/pull/25819#discussion_r4311210889
##########
datafusion/catalog-listing/src/table.rs:
##########
@@ -1061,9 +1075,18 @@ impl ListingTable {
if !file_groups.is_empty() {
file_groups.resize_with(file_group_count, ||
FileGroup::new(vec![]));
}
- let file_groups =
+ let mut file_groups =
self.filter_declared_file_groups_by_partition_filters(file_groups,
filters)?;
+ if let LogicalPartitioning::Range(range) = output_partitioning {
+ prune_range_file_groups(
+ &mut file_groups,
+ range,
+ range_filters,
+ &self.table_schema,
+ );
+ }
Review Comment:
> Mm, looks like that is covered here: #26441
Thanks for checking #26441! It addresses the related ordering issue for
regular scans by pruning with collected statistics before grouping. It
currently skips declared layouts, though, so moving Range pruning ahead of
statistics collection would still be a separate improvement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
