lyne7-sc commented on PR #24444: URL: https://github.com/apache/datafusion/pull/24444#issuecomment-5344174159
> One thought for further improvement: it seems more efficient to handle the aggregate filter entirely during preprocessing. For example, we could first evaluate the filter, then use the resulting selection for late materialization of the group keys and aggregate arguments. > > I think we currently still pass the filter through because the existing `GroupsAccumulator ` API is able to handle the filter, but that may no longer be necessary once filtering has already been applied upstream. This makes sense. It looks like the non-grouped Accumulator path already filters the batch before evaluating its arguments. For grouped aggregation, I think we would still need to create groups from all rows, then pass only the selected argument values and their corresponding group indices to GroupsAccumulator. That should allow us to avoid passing the filter through. I’ll take a closer look at this approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
