englefly opened a new pull request, #69074:
URL: https://github.com/apache/doris/pull/69074
### What problem does this PR solve?
Problem Summary:
RQG with fe_debug=true stably trips the nullable consistency check in
AdjustNullable when a set operation is combined with uniform constant
elimination:
SQL explain failed: AdjustNullable convert slot 2004-04-28#158
from not-nullable to nullable.
AdjustNullable.visitLogicalSetOperation recomputed the output nullable of
every set operation with the union rule (nullable when any child column is
nullable). That rule is only correct for UNION. A row is in the result of an
INTERSECT only when it is in every child, so the column can be null only when
every child column can be null; the result of an EXCEPT is a subset of the
left child(child(0)), so only the left child can make it null. Applying the
union rule to them marked output columns that provably can not be null as
nullable, and every not-nullable reference the upper plan had derived was
widened afterwards, which is what the debug check reported.
Both reported RQG queries have the same shape: one branch of the set
operation
is empty (partition pruning / an unsatisfiable predicate), the constant
branch
makes the remaining set operation output a uniform constant, constant
propagation turns the group-by key into a literal and
EliminateGroupByKeyByUniform rewrites the distinct aggregate into Limit 1, so
the result sink/upper project directly references the set operation output
slot that AdjustNullable widened at the end of the rewrite.
Case 11000258898: select '2004-04-28', ... intersect ... union distinct
(select ... where dt = const intersect select not_null_col ... intersect
select not_null_col ...); the first column of the second INTERSECT branch can
not be null because two of its children are built from not-null columns, so
the UNION output is not nullable either.
Case 11000259585: (select 'vZRgkfajsD', 'cKToNmsxok' except ...) union
distinct ... except ...; the second column of the constant branch can only be
'cKToNmsxok', so the EXCEPT output is not nullable.
Fix:
AdjustNullable computes the output nullable per set operation semantics
(union: or, intersect: and, except: left child) and keeps a column nullable
when a child output can not be found (fail safe).
Reproduction: the new FE unit test AdjustNullableSetOperationTest covers the
nullable semantics of intersect/union/except and both RQG query shapes; three
of its cases fail without this change.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- AdjustNullableSetOperationTest (8 cases, 3 of them fail before the fix)
- plus related FE unit test classes (set operation, nullability, fd,
uniform, constant propagation, column pruning) rerun without new
failures.
- Behavior changed: Yes. The output column of an INTERSECT/EXCEPT is no
longer
reported as nullable when it provably can not be null, which also keeps the
not null attribute of the result schema.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
