suremarc opened a new issue, #11362:
URL: https://github.com/apache/datafusion/issues/11362
### Describe the bug
After PR #10434, there is a slight regression in eliminating certain sorts
during physical planning. In particular, consider the following case:
```
SortExec: [c ASC]
FilterExec: b = 'A'
CustomTableProviderExec: output_ordering=[a ASC, b ASC, c ASC],
equal_conditions=[a=f(c)]
```
`f` is known to be monotonic in `c`. Hence we know that `[a ASC, c ASC]` is
equivalent to `c ASC`. (See discussion in #9812.)
We are unable to eliminate sorting in this case. This is because after PR
#10434, adding constants to the eq properties does not leave them in a
normalized state. This was not caught in tests, as the tests add constants
_before_ adding equal conditions.
In practice, however, physical plans like the one above add constants
_after_ adding equal conditions.
### To Reproduce
The simplest way to reproduce this behavior, short of implementing a custom
table provider, is to modify [this existing
test](https://github.com/apache/datafusion/blob/e65c3e919855c9977cf4d80c0630ee26b7fd03ee/datafusion/physical-expr/src/equivalence/properties.rs#L2382)
to add constants _after_ adding the equal conditions. On the current version
of `main` branch this will cause the test to fail.
### Expected behavior
Equivalence properties should be the same regardless of what order the equal
conditions, constants, etc. are added. (In particular the test should pass
either way.)
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
