Revanth14 opened a new issue, #18701: URL: https://github.com/apache/iceberg/issues/18701
### Feature Request / Improvement ## Problem `Evaluator` compares through literal comparators that order null first, so `eq` is false and `notEq` is true for a null value. `in` returns false for null, so `notIn` is true. Under SQL three-valued logic, `x <> v` and `x NOT IN (...)` are unknown when `x` is null, and a filter does not keep the row, so rows with a null value match `notEq` and `notIn` filters in `Evaluator` but not in SQL engines. ## Proposal Treat a comparison, `in` or `notIn` on a null value as unknown, and keep a row only when the filter is true, so that `not` of an unknown stays unknown. This would make `Evaluator` agree with SQL engines on null rows. ## Open question Whether `Evaluator` is meant to follow SQL semantics here, or is intentionally two-valued for callers that rely on the current behaviour. ## Context iceberg-go changed its row filter to SQL semantics for `In` and `NotIn` in apache/iceberg-go#2356; `NotEqual` already followed them. ### Query engine None ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
