yashrb24 opened a new issue, #25018:
URL: https://github.com/apache/datafusion/issues/25018
### Describe the bug
When a `WHERE` condition limits a `GROUP BY` column to one value, we have
the pieces to know that the column is fixed. However, aggs currently treats
that column as if a change in its value could show that earlier groups are
complete.
Since the column is fixed because of the filter, it cannot provide such a
boundary.
eg:
```sql
SELECT key, market, COUNT(*)
FROM t
WHERE market = 'US'
GROUP BY key, market;
```
Today we currently report the agg as partially ordered using `market`. For
large inputs, this blocks releasing finished groups early
### To Reproduce
```sql
EXPLAIN
WITH t(key, market) AS (
VALUES
(1, 'US'),
(2, 'US'),
(1, 'US')
)
SELECT key, market, COUNT(*)
FROM t
WHERE market = 'US'
GROUP BY key, market;
```
As of today, this produces
```text
AggregateExec: mode=Partial,
gby=[key@0 as key, market@1 as market],
ordering_mode=PartiallySorted([1])
```
Every row has `market = 'US'`, so this column never changes and cannot show
that a group has finished.
### Expected behavior
Ideally should ignore fixed grouping columns when deciding whether earlier
groups are complete. For the example above, agg should not use `market` as a
group completion boundary. If another grouping column is genuinely ordered, we
should still use that column
### Additional context
### Additional context
#24697 is somewhat related to this fix. It separates input ordering from the
question of whether a group is complete. A fixed column can be considered
ordered, but it should not be used as that a group is complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
