github-actions[bot] commented on code in PR #70516:
URL: https://github.com/apache/doris/pull/70516#discussion_r4655992340


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterPushDownVisitor.java:
##########
@@ -308,6 +318,25 @@ && canPushThroughJoinChild(join, false, ctx)) {
                 }
                 if (newTarget != null && newTarget.getInputSlots().size() == 1
                         && !newTarget.equals(ctx.srcExpr)) {
+                    if (expandByOuterJoin) {

Review Comment:
   [P1] Stop expanded runtime filters at sensitive window expressions
   
   `visitPhysicalWindow` pushes through whenever the probe slot is a common 
partition key, without checking the window expressions or order keys. The new 
outer-join branch can therefore filter the window input before a side-effecting 
expression runs:
   
   ```text
   InnerJoin(L.k = E.k)
     LeftOuterJoin(L.k = R.k)
       Scan L
       Window(COUNT(assert_true(R.v > 10, 'bad')) OVER (PARTITION BY R.k))
         Scan R
     Scan E={2}
   ```
   
   With `L={2}`, `E={2}`, and `R={(1,10),(2,20)}`, the expanded RF removes the 
key-1 partition before the window evaluates `assert_true` and suppresses `bad`; 
without expansion the window evaluates it and fails. Add a volatile/NoneMovable 
barrier for window expressions (or conservatively stop this traversal) and a 
regression for the non-preserved window path.



##########
fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterPushDownVisitor.java:
##########
@@ -308,6 +318,25 @@ && canPushThroughJoinChild(join, false, ctx)) {
                 }
                 if (newTarget != null && newTarget.getInputSlots().size() == 1
                         && !newTarget.equals(ctx.srcExpr)) {
+                    if (expandByOuterJoin) {
+                        Plan preserved = join.getJoinType().isLeftOuterJoin() 
? leftNode : rightNode;
+                        Plan nonPreserved = 
join.getJoinType().isLeftOuterJoin() ? rightNode : leftNode;

Review Comment:
   [P1] Stop expanded runtime filters at sensitive sorts
   
   The new outer-join branch can recurse through a retained 
`PhysicalQuickSort`, but the visitor has no sort barrier:
   
   ```text
   InnerJoin(L.k = E.k)
     LeftOuterJoin(L.k = R.k)
       Scan L
       QuickSort(order by assert_true(R.v > 10, 'bad'))
         Scan R
     Scan E={2}
   ```
   
   With `L={2}`, `E={2}`, and `R={(1,10),(2,20)}`, the expanded RF removes key 
1 before the order expression runs and the query succeeds; without expansion 
the sort raises `bad`. A derived-table sort can be retained when sort 
elimination is disabled. Add a QuickSort barrier for volatile/NoneMovable order 
keys (or conservatively fence sort traversal) and a regression with sort 
elimination disabled.



##########
fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterPushDownVisitor.java:
##########
@@ -293,11 +297,17 @@ && canPushThroughJoinChild(join, false, ctx)) {
             pushed |= rightNode.accept(this, ctx);
         }
 
-        // Expand through join conditions at non-builder inner/semi joins
+        // Outer joins only propagate a parent's ordinary join RF from the 
preserved side.
+        // Decoupled RFs do not carry the parent equality here, so leave them 
out of this expansion.
+        boolean expandByOuterJoin = !join.isMarkJoin()

Review Comment:
   [P1] Stop expanded runtime filters at sensitive aggregates
   
   The new outer-join branch can recurse through a `PhysicalHashAggregate` when 
the probe is its direct group key, but the visitor has no aggregate barrier:
   
   ```text
   InnerJoin(L.k = E.k)
     LeftOuterJoin(L.k = R.k)
       Scan L
       Project(R.k, c)
         HashAggregate(group by R.k,
                       COUNT(assert_true(R.v > 10, 'bad')) AS c)
           Scan R
     Scan E={2}
   ```
   
   With `L={2}`, `E={2}`, and `R={(1,10),(2,20)}`, the expanded RF removes the 
key-1 input before the aggregate argument runs and the query succeeds; without 
expansion it raises `bad`. Add a volatile/NoneMovable aggregate barrier (or 
conservatively stop traversal through such aggregates) and a regression for 
this non-preserved aggregate path.



##########
fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterPushDownVisitor.java:
##########
@@ -293,11 +297,17 @@ && canPushThroughJoinChild(join, false, ctx)) {
             pushed |= rightNode.accept(this, ctx);
         }
 
-        // Expand through join conditions at non-builder inner/semi joins
+        // Outer joins only propagate a parent's ordinary join RF from the 
preserved side.
+        // Decoupled RFs do not carry the parent equality here, so leave them 
out of this expansion.
+        boolean expandByOuterJoin = !join.isMarkJoin()
+                && ctx.exprOrder >= 0 && ctx.builderNode instanceof 
PhysicalHashJoin
+                && (join.getJoinType().isLeftOuterJoin() || 
join.getJoinType().isRightOuterJoin());
+        // Expand through join conditions at non-builder joins.
         if (!join.equals(ctx.builderNode)
                 && ConnectContext.get() != null
                 && 
ConnectContext.get().getSessionVariable().expandRuntimeFilterByInnerJoin
-                && (join.getJoinType() == JoinType.INNER_JOIN || 
join.getJoinType().isSemiJoin())) {
+                && (join.getJoinType() == JoinType.INNER_JOIN || 
join.getJoinType().isSemiJoin()
+                        || expandByOuterJoin)) {

Review Comment:
   [P1] Stop expanded runtime filters at volatile/non-movable filters
   
   The new outer-join branch derives `R.k` and calls the generic visitor on the 
non-preserved subtree, but `PhysicalFilter` has no barrier here 
(`PlanVisitor.visitPhysicalFilter` recurses to its child). For example:
   
   ```text
   InnerJoin(L.k = E.k)
     LeftOuterJoin(L.k = R.k)
       Scan L
       Filter(assert_true(R.v > 10, 'bad'))
         Scan R
     Scan E={2}
   ```
   
   With `L={2}`, `E={2}`, and `R={(1,10),(2,20)}`, expansion attaches the RF 
for `E.k` to `R.k`, so the `k=1` row is removed before `assert_true` and the 
query succeeds. Without expansion, the filter evaluates that row and raises 
`bad`. Add a `containsVolatileOrNoneMovableExpression()` filter barrier and a 
regression for this filtered non-preserved path.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to