morrySnow opened a new pull request, #24232:
URL: https://github.com/apache/doris/pull/24232
pick from master:
PR: #23973
commit-id: b2ca28139553cacd0f5f2e492e30d3a37ce26bed
when output meet order by not meet distribution. we use a trick way to do
enforce by set current output to any. but when we do enforce later, we still
use the old output. So when we do choose best plan, we could not find the older
output's plan, since we have replace it by any. For example:
```
lowest Plan(cost, properties, plan, childrenRequires)
18.0 ANY
id:138#4 cost=0 [0/0/0/] estRows=4 children=[@0 ]
(plan=PhysicalWindow[139]@4 ( windowFrameGroup=(Funcs=[row_number()
WindowSpec(PARTITION BY b#1, a#0 ROWS BETWEEN UNBOUNDED_PRECEDING AND
CURRENT_ROW) AS `r1`#2], PartitionKeys=[b#1, a#0], OrderKeys=[],
WindowFrame=WindowFrame(ROWS, UNBOUNDED_PRECEDING, CURRENT_ROW)),
requiredProperties=[DistributionSpecHash ( orderedShuffledColumns=[1, 0],
shuffleType=REQUIRE, tableId=-1, selectedIndexId=-1, partitionIds=[],
equivalenceExprIds=[[1], [0]], exprIdToEquivalenceSet={0=1, 1=0} ) Order: ([b#1
asc, a#0 asc])], stats=null ))
[DistributionSpecHash ( orderedShuffledColumns=[0],
shuffleType=NATURAL, tableId=3547296, selectedIndexId=3547297,
partitionIds=[3547295], equivalenceExprIds=[[0]], exprIdToEquivalenceSet={0=0}
) Order: ([b#1 asc, a#0 asc])]
32.01171875 DistributionSpecHash ( orderedShuffledColumns=[1],
shuffleType=REQUIRE, tableId=-1, selectedIndexId=-1, partitionIds=[],
equivalenceExprIds=[[1]], exprIdToEquivalenceSet={1=0} ) Order: ([b#1 asc])
id:161#4 cost=14 [4/4/4/] estRows=4 children=[@4 ]
(plan=PhysicalQuickSort[162]@4 ( orderKeys=[b#1 asc], phase=LOCAL_SORT,
stats=null ))
[DistributionSpecHash ( orderedShuffledColumns=[0],
shuffleType=NATURAL, tableId=3547296, selectedIndexId=3547297,
partitionIds=[3547295], equivalenceExprIds=[[0]], exprIdToEquivalenceSet={0=0}
) Order: ([b#1 asc, a#0 asc])]
32.01171875 DistributionSpecHash ( orderedShuffledColumns=[1],
shuffleType=EXECUTION_BUCKETED, tableId=-1, selectedIndexId=-1,
partitionIds=[], equivalenceExprIds=[[1]], exprIdToEquivalenceSet={1=0} )
Order: ([b#1 asc])
id:161#4 cost=14 [4/4/4/] estRows=4 children=[@4 ]
(plan=PhysicalQuickSort[162]@4 ( orderKeys=[b#1 asc], phase=LOCAL_SORT,
stats=null ))
[DistributionSpecHash ( orderedShuffledColumns=[1],
shuffleType=EXECUTION_BUCKETED, tableId=-1, selectedIndexId=-1,
partitionIds=[], equivalenceExprIds=[[1]], exprIdToEquivalenceSet={1=0} )
Order: ([])]
18.01171875 DistributionSpecHash ( orderedShuffledColumns=[1],
shuffleType=EXECUTION_BUCKETED, tableId=-1, selectedIndexId=-1,
partitionIds=[], equivalenceExprIds=[[1]], exprIdToEquivalenceSet={1=0} )
Order: ([])
id:157#4 cost=0 [0/0/0/] estRows=4 children=[@4 ]
(plan=PhysicalDistribute[158]@4 ( distributionSpec=DistributionSpecHash (
orderedShuffledColumns=[1], shuffleType=EXECUTION_BUCKETED, tableId=-1,
selectedIndexId=-1, partitionIds=[], equivalenceExprIds=[[1]],
exprIdToEquivalenceSet={1=0} ), stats=null ))
[DistributionSpecHash ( orderedShuffledColumns=[0],
shuffleType=NATURAL, tableId=3547296, selectedIndexId=3547297,
partitionIds=[3547295], equivalenceExprIds=[[0]], exprIdToEquivalenceSet={0=0}
) Order: ([b#1 asc, a#0 asc])]
```
the last one require a natural shuffle type property from this group. but
this property already been removed when we do
enforceDistributionButMeetSort. So, such exception will be thrown
```
Caused by: org.apache.doris.nereids.exceptions.AnalysisException: Failed to
choose best plan
at
org.apache.doris.nereids.NereidsPlanner.chooseBestPlan(NereidsPlanner.java:340)
~[classes/:?]
at
org.apache.doris.nereids.NereidsPlanner.chooseBestPlan(NereidsPlanner.java:323)
~[classes/:?]
... 18 more
Caused by: org.apache.doris.nereids.exceptions.AnalysisException:
lowestCostPlans with physicalProperties(DistributionSpecHash (
orderedShuffledColumns=[0], shuffleType=NATURAL, tableId=3547296,
selectedIndexId=3547297, partitionIds=[3547295], equivalenceExprIds=[[0]],
exprIdToEquivalenceSet={0=0} ) Order: ([b#1 asc, a#0 asc])) doesn't exist in
root group
at
org.apache.doris.nereids.NereidsPlanner.lambda$chooseBestPlan$1(NereidsPlanner.java:318)
~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:408) ~[?:?]
at
org.apache.doris.nereids.NereidsPlanner.chooseBestPlan(NereidsPlanner.java:317)
~[classes/:?]
at
org.apache.doris.nereids.NereidsPlanner.chooseBestPlan(NereidsPlanner.java:323)
~[classes/:?]
... 18 more
```
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
