viirya commented on PR #19857:
URL: https://github.com/apache/datafusion/pull/19857#issuecomment-6021823612
Thanks @comphead. ab3120cc9 addresses the review:
**Tests**
- `null_aware_multi_column.slt` now covers:
- `IS [NOT] NULL` over a tuple `IN`
- subqueries whose top is `DISTINCT`, `GROUP BY` or `ORDER BY ... LIMIT`
- `Utf8` elements
- a `LeftMark` tuple with an `i.k < o.k` correlation
- the session time zone case
- placeholders over two same-named subquery columns
- Added `fuzz_cases/null_aware_join_fuzz.rs`. It compares scalar and
multi-column `NOT IN` / `IN`, in filter, `OR` and value contexts, with and
without equality and non-equality correlations, against `NOT EXISTS` / `EXISTS`
formulations of the same three-valued logic. It sweeps random tables with NULLs
on both sides across batch sizes and partition counts.
**Docs**
- `subqueries.md` now documents the tuple form, allows one subquery column
per tuple element in both places that said "a single column", and explains that
`(1, NULL) IN (SELECT 1, NULL)` is NULL under row comparison while the list
form is `true`. The examples are pinned in the SLT.
**Follow-ups**
- Filed #26088 (NOT NULL elements as scope keys) and #26089 (a multi-key
null-aware `RightAnti` that builds on the subquery side), linked from the doc
comment on `HashJoinExec::null_aware_value_keys`.
- Q10 to Q12 at two sizes (mean of three runs each):
| query | `NAJ_ROWS=10000` | `NAJ_ROWS=20000` |
|---|---|---|
| Q10, no NULLs | 0.32 ms | 0.38 ms |
| Q11, 1% NULLs in the subquery | 3.64 ms | 8.07 ms |
| Q12, 1% NULLs in the outer side | 0.56 ms | 0.76 ms |
**Nit**
- Placeholder inference now uses
`Column::from(subquery_schema.qualified_field(i))`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
