847850277 commented on issue #23410: URL: https://github.com/apache/datafusion/issues/23410#issuecomment-5248165284
> I can't reproduce this on `main` @ `308e212` — both DataFusion and PostgreSQL 14 return an empty result for the query as written. > > I checked four ways: > > 1. **The exact reproduction above** (in-memory tables) — empty. > 2. **Ten variants around it** — filter placed before vs. after the subquery, a literal `a NOT IN (1, NULL)` list, a subquery mixing NULL and non-NULL values, a correlated subquery, and `(a != 1 OR a IS NULL)`. All ten match PostgreSQL exactly. > 3. **Execution settings** — `target_partitions` at 1, 8 and 32, plus `filter_null_join_keys = true` and `repartition_joins = false`. All empty. (I checked partition counts because [pg_compat_window.slt fails when target_partitions >= 32 #5082](https://github.com/apache/datafusion/issues/5082) showed `pg_compat_window.slt` behaving differently at `target_partitions >= 32`.) > 4. **CSV-backed external tables**, 200 rows, with the fixture asserted to contain exactly one NULL row — 0 rows. > > The plan does still push the filter beneath the null-aware anti join, which I believe is the shape flagged in the report: > > ``` > logical_plan > 01)Projection: t.id > 02)--LeftAnti Join: t.a = __correlated_sq_1.k null_aware > 03)----Filter: t.a != Int32(1) > 04)------TableScan: t projection=[id, a] > 05)----SubqueryAlias: __correlated_sq_1 > 06)------TableScan: u projection=[k] > ``` > > I couldn't find a PR referencing this issue, so I don't know what changed in between. Is there context missing from the reproduction — a specific version, or a table provider other than in-memory? Happy to re-test if so. Hi，It seems to have been solved when this pr merged <img width="1133" height="356" alt="Image" src="https://github.com/user-attachments/assets/fb7ade78-4915-425c-a714-2304ae51e598"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
