adriangb opened a new issue, #24711: URL: https://github.com/apache/datafusion/issues/24711
Follow-up from #24526. That PR's compact path handles positive `IN` with all-non-NULL string literals only. Both exclusions look right as shipped, but the review discussion suggested they're two separate problems with different answers. **NULL-containing lists.** `x IN (a, b, NULL)` is TRUE or UNKNOWN, never FALSE. Dropping the NULL literal and pruning on the rest is fine for filtering, since UNKNOWN and FALSE both exclude a row — but it turns UNKNOWN into FALSE, and `identify_fully_matched_row_groups` inverts the predicate and reads "no row can match" as "every row matches the negation". That implication needs the predicate to be two-valued. The compensation there is an `OR IsNull(column)` disjunct per nullable column, which assumes a NULL column value is the only route to UNKNOWN; a NULL literal inside the list isn't visible to it. Worth noting the compensation is adequate today — the per-value path rewrites the NULL element into a NULL-valued comparison and degrades to "always keep", so no incorrect full-match claim appears reachable on current `main`. So this reads as "what would need to be true first" rather than a live bug. **`NOT IN`.** A separate problem: negating interval intersection isn't a valid pruning rule, since an overlapping interval doesn't prove every row belongs to the excluded set. Supporting `NOT IN` would need a different rule rather than a negation of this one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
