englefly opened a new pull request, #69075:
URL: https://github.com/apache/doris/pull/69075
### What problem does this PR solve?
Problem Summary:
EliminateUnnecessaryProject removed a project only by comparing the exprIds
of
its output with the exprIds of its child:
project.getOutputExprIdBitSet().equals(project.getChildrenOutputExprIdBitSet())
That equality does not mean the project is a pass-through.
ConstantPropagation
replaces a slot reference with a literal but keeps the exprId of the replaced
slot (ConstantPropagation#visitLogicalAggregate:
new Alias(expr.getExprId(), replacedExpr, expr.getName())), so such a project
outputs the same exprIds as its child while its slots describe not nullable
literals. RQG case 11000258898 reaches exactly this shape:
BEFORE:
LogicalResultSink[61] ( outputExprs=[2004-04-28#55, 2014-07-07#56] )
+--LogicalLimit ( limit=1, offset=0, phase=GLOBAL )
+--LogicalProject[260] ( projects=['2011-11-15 13:33:05.000' AS
`2004-04-28`#55,
'2011-11-15 13:33:05.000' AS
`2014-07-07`#56] )
+--LogicalIntersect ( qualifier=DISTINCT, outputs=[2004-04-28#55,
2014-07-07#56], ... )
AFTER:
LogicalResultSink[61] ( outputExprs=[2004-04-28#55, 2014-07-07#56] )
+--LogicalLimit ( limit=1, offset=0, phase=GLOBAL )
+--LogicalIntersect ( qualifier=DISTINCT, outputs=[2004-04-28#55,
2014-07-07#56], ... )
The project is dropped because its output exprIds {55, 56} equal the child
intersect's output exprIds {55, 56}, although the project's #55 is the not
nullable literal cast('2011-11-15 13:33:05.000' as TEXT) while the
intersect's
#55 is nullable at schema level (its first branch reads the nullable column
col_datetime_3__undef_signed). After the project is removed the upper plan
reads the nullable slot again; those references had already been synced to
the
not nullable project slot during the rewrite, so the last AdjustNullable pass
widens them and fe_debug=true reports
"AdjustNullable convert slot 2004-04-28#55 from not-nullable to nullable".
Dropping such a project also loses the data type carried by its slots.
Fix:
EliminateUnnecessaryProject only removes a project when every project item is
a plain SlotReference of the child, i.e. a real pass-through. The reason why
comparing exprIds is not enough is documented next to the check.
### Release note
None
### Check List (For Author)
- Test: Unit Test
-
EliminateUnnecessaryProjectTest#testKeepProjectThatReplacesSlotsWithLiterals
builds such a project and fails before this change with
"expected node type UNKNOWN but got LOGICAL_OLAP_SCAN".
- EliminateUnnecessaryProjectTest, ColumnPruningTest,
MergeProjectableTest,
PushProjectIntoUnionTest, SetOperationOutputMappingTest,
ConstantPropagationTest and SqlPlanSuiteTest rerun without new
failures.
- Behavior changed: Yes. A project that replaces output slots with literals
is
kept, so the not null and data type attributes of those slots survive.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
