jayzhan211 commented on code in PR #24801:
URL: https://github.com/apache/datafusion/pull/24801#discussion_r4063284771
##########
datafusion/optimizer/src/push_down_filter.rs:
##########
@@ -1141,6 +1141,32 @@ impl OptimizerRule for PushDownFilter {
result.map_data(|plan| Ok(with_filters(keep_predicates, plan)))
}
LogicalPlan::Join(join) => push_down_join(join,
Some(filter.predicate)),
+ LogicalPlan::AsOfJoin(mut join) => {
Review Comment:
Follow-up is fine: a pushed left-only predicate on an equality key (`ON
l.grp = r.grp`, `WHERE l.grp = 'A'`) can be mirrored to the right input as
`r.grp = 'A'`. Right rows in other groups can never match a surviving left row,
so null padding is unaffected. It is the same idea as `infer_join_predicates`
for regular joins. Q08 has no key, so a keyed benchmark would be needed to show
the gain.
I reproduced Q08 on Apple silicon (release build, interleaved runs, 7
iterations): 121.5–127.3 ms → 18.0 ms median, about 6.9x.
##########
datafusion/sqllogictest/test_files/asof_join.slt:
##########
@@ -256,6 +256,64 @@ physical_plan
04)--SortExec: expr=[grp@0 ASC, ts@1 ASC], preserve_partitioning=[false]
05)----DataSourceExec: partitions=1, partition_sizes=[1]
+# Deterministic predicates that reference only the left input can run before
Review Comment:
Both new tests are `EXPLAIN`-only. Nothing executes a query with the left
filter below the join next to a null-padded right predicate, so a bug in which
conjunct lands where would only change plan text. Please add result queries. I
ran these on this branch and they pass:
```sql
query IT
SELECT l.id, r.val
FROM asof_left l
ASOF JOIN asof_right r
MATCH_CONDITION (l.ts >= r.ts)
ON l.grp = r.grp
WHERE l.id > 1 AND r.val <> 'a2' AND l.id <> length(r.val)
ORDER BY l.id;
----
3 a6
4 b1
5 b6
query IT
SELECT l.id, r.val
FROM asof_left l
ASOF JOIN asof_right r
MATCH_CONDITION (l.ts >= r.ts)
ON l.grp = r.grp
WHERE l.grp IS NULL OR l.ts IS NULL OR l.id = 1
ORDER BY l.id;
----
1 NULL
6 NULL
7 NULL
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
