[
https://issues.apache.org/jira/browse/CALCITE-7450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Darpan Lunagariya (e6data computing) updated CALCITE-7450:
----------------------------------------------------------
Description:
In ValuesReduceRule, when a filter condition on LogicalValues contains a
function call that RexExecutorImpl cannot reduce to a RexLiteral (e.g., a UDF
with no Janino CallImplementor, or a non-deterministic function like RAND()),
the unreduced RexCall is passed to isAlwaysTrue(), which always returns false
for any RexCall. This causes the rule to incorrectly drop the tuple.
The project expression path already handles this correctly — it checks whether
each reduced expression is a RexLiteral and bails out if not (lines 203-209).
The filter condition path lacks this same guard.
Below are the relevant snippets from that rule:
*For project:*
{code:java}
if (projectExprs != null) {
++changeCount;
final ImmutableList.Builder<RexLiteral> tupleBuilder =
ImmutableList.builder();
for (; i < fieldsPerRow; ++i) {
final RexNode reducedValue =
reducibleExps.get((row * fieldsPerRow) + i);
if (reducedValue instanceof RexLiteral) {
tupleBuilder.add((RexLiteral) reducedValue);
} else if (RexUtil.isNullLiteral(reducedValue, true)) {
tupleBuilder.add(rexBuilder.makeNullLiteral(reducedValue.getType()));
} else {
return;
}
}
valuesList = tupleBuilder.build();
} else {
valuesList = values.getTuples().get(row);
}
{code}
*For filter:*
{code:java}
if (conditionExpr != null) {
final RexNode reducedValue =
reducibleExps.get((row * fieldsPerRow) + i);
++i;
if (!reducedValue.isAlwaysTrue()) {
++changeCount;
continue;
}
}
{code}
For filter conditions, we just check
{code:java}
!reducedValue.isAlwaysTrue(){code}
which will always be true in case of RexCall and that tuple will be dropped
incorrectly, instead, we should just return from that rule, as done in reducing
the project expression.
Here is one example:
{code:sql}
SELECT * FROM (VALUES (0, 1, 2), (3, 4, 5)) AS t(a, b, c) WHERE RAND(t.a) > 0.5
{code}
For this query,
{code:java}
// RelNode tree after conversion to RelNode from SqlNode:
LogicalProject(A=[$0], B=[$1], C=[$2])
LogicalFilter(condition=[>(RAND($0), CAST(0.5:DECIMAL(2, 1)):DOUBLE NOT
NULL)])
LogicalValues(tuples=[[{ 0, 1, 2 }, { 3, 4, 5 }]])
// RelNode tree applying the rule:
LogicalValues(tuples=[[]])
{code}
was:
When we have *custom functions* defined and use them on LogicalValues, the
ValuesReduceRule will drop the tuples in a wrong way, which it should not do.
The project expression reduction handles it correctly and it specifically
checks that after reduction, did the RexCall reduced into RexLiteral or not, if
not, then we return at that time, but this is not handled in filter expression
reduction.
Below are the relevant snippets from that rule:
*For project:*
{code:java}
if (projectExprs != null) {
++changeCount;
final ImmutableList.Builder<RexLiteral> tupleBuilder =
ImmutableList.builder();
for (; i < fieldsPerRow; ++i) {
final RexNode reducedValue =
reducibleExps.get((row * fieldsPerRow) + i);
if (reducedValue instanceof RexLiteral) {
tupleBuilder.add((RexLiteral) reducedValue);
} else if (RexUtil.isNullLiteral(reducedValue, true)) {
tupleBuilder.add(rexBuilder.makeNullLiteral(reducedValue.getType()));
} else {
return;
}
}
valuesList = tupleBuilder.build();
} else {
valuesList = values.getTuples().get(row);
}
{code}
*For filter:*
{code:java}
if (conditionExpr != null) {
final RexNode reducedValue =
reducibleExps.get((row * fieldsPerRow) + i);
++i;
if (!reducedValue.isAlwaysTrue()) {
++changeCount;
continue;
}
}
{code}
For filter conditions, we just check
{code:java}
!reducedValue.isAlwaysTrue(){code}
which will always be true in case of RexCall and that tuple will be dropped
incorrectly, instead, we should just return from that rule, as done in reducing
the project expression.
Here is one example:
{code:sql}
SELECT * FROM (VALUES (0, 1, 2), (3, 4, 5)) AS t(a, b, c) WHERE
CUSTOM_FUNC(t.a) > 2
{code}
I have just registered CUSTOM_FUNC in operator table, but I did not register
Janino CallImplementor. Possibility is that this function is executed at
runtime and some application just expect calcite to produce the Plan for it.
For this query,
{code:java}
// RelNode tree after conversion to RelNode from SqlNode:
LogicalProject(A=[$0], B=[$1], C=[$2])
LogicalFilter(condition=[>(CUSTOM_FUNC($0), 2)])
LogicalValues(tuples=[[{ 0, 1, 2 }, { 3, 4, 5 }]])
// RelNode tree applying the rule:
LogicalValues(tuples=[[]])
{code}
> ValuesReduceRule dropping LogicalValues incorrectly
> ---------------------------------------------------
>
> Key: CALCITE-7450
> URL: https://issues.apache.org/jira/browse/CALCITE-7450
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.41.0, 1.42.0
> Reporter: Darpan Lunagariya (e6data computing)
> Priority: Minor
> Labels: pull-request-available
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In ValuesReduceRule, when a filter condition on LogicalValues contains a
> function call that RexExecutorImpl cannot reduce to a RexLiteral (e.g., a UDF
> with no Janino CallImplementor, or a non-deterministic function like RAND()),
> the unreduced RexCall is passed to isAlwaysTrue(), which always returns false
> for any RexCall. This causes the rule to incorrectly drop the tuple.
>
> The project expression path already handles this correctly — it checks
> whether each reduced expression is a RexLiteral and bails out if not (lines
> 203-209). The filter condition path lacks this same guard.
> Below are the relevant snippets from that rule:
> *For project:*
> {code:java}
> if (projectExprs != null) {
> ++changeCount;
> final ImmutableList.Builder<RexLiteral> tupleBuilder =
> ImmutableList.builder();
> for (; i < fieldsPerRow; ++i) {
> final RexNode reducedValue =
> reducibleExps.get((row * fieldsPerRow) + i);
> if (reducedValue instanceof RexLiteral) {
> tupleBuilder.add((RexLiteral) reducedValue);
> } else if (RexUtil.isNullLiteral(reducedValue, true)) {
> tupleBuilder.add(rexBuilder.makeNullLiteral(reducedValue.getType()));
> } else {
> return;
> }
> }
> valuesList = tupleBuilder.build();
> } else {
> valuesList = values.getTuples().get(row);
> }
> {code}
> *For filter:*
> {code:java}
> if (conditionExpr != null) {
> final RexNode reducedValue =
> reducibleExps.get((row * fieldsPerRow) + i);
> ++i;
> if (!reducedValue.isAlwaysTrue()) {
> ++changeCount;
> continue;
> }
> }
> {code}
>
> For filter conditions, we just check
> {code:java}
> !reducedValue.isAlwaysTrue(){code}
> which will always be true in case of RexCall and that tuple will be dropped
> incorrectly, instead, we should just return from that rule, as done in
> reducing the project expression.
>
> Here is one example:
> {code:sql}
> SELECT * FROM (VALUES (0, 1, 2), (3, 4, 5)) AS t(a, b, c) WHERE RAND(t.a) >
> 0.5
> {code}
> For this query,
> {code:java}
> // RelNode tree after conversion to RelNode from SqlNode:
> LogicalProject(A=[$0], B=[$1], C=[$2])
> LogicalFilter(condition=[>(RAND($0), CAST(0.5:DECIMAL(2, 1)):DOUBLE NOT
> NULL)])
> LogicalValues(tuples=[[{ 0, 1, 2 }, { 3, 4, 5 }]])
> // RelNode tree applying the rule:
> LogicalValues(tuples=[[]])
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
