LuciferYang opened a new issue, #3292: URL: https://github.com/apache/iceberg-rust/issues/3292
### Apache Iceberg Rust version main (0.10.x) ### Describe the bug `Predicate::bind` rewrites `IS NULL` on a required field to `AlwaysFalse` and `IS NOT NULL` to `AlwaysTrue` from the leaf field's own `required` flag alone, without checking whether its ancestor structs are nullable: https://github.com/apache/iceberg-rust/blob/86d661880/crates/iceberg/src/expr/predicate.rs#L388-L398 The spec says a null parent struct implies its leaves are null: "if a parent struct column is null it implies the leaf column is null" ([format/spec.md](https://github.com/apache/iceberg/blob/main/format/spec.md)). So the rewrite only holds when every ancestor of the field is also required. Java applies it that way: it walks the ancestors and only shortcuts when all of them are required (`UnboundPredicate` in the Java library). With `person` optional and `person.age` required, `person.age IS NULL` binds to `AlwaysFalse`, so a scan over a file where some row has a null `person` returns no rows instead of the rows whose `person` is null. `IS NOT NULL` has the mirror problem: it binds to `AlwaysTrue` and keeps rows it should drop. ### To Reproduce Schema `person: optional struct<age: required int>`, and a file with one row whose `person` is null. `Reference::new("person.age").is_null().bind(schema, false)` returns `AlwaysFalse`, and a scan with that predicate drops the null-`person` row instead of keeping it. ### Expected behavior `IS NULL` on a required field should rewrite to `AlwaysFalse` (and `IS NOT NULL` to `AlwaysTrue`) only when the field and all of its ancestor structs are required. When any ancestor is optional, keep the predicate and evaluate it. ### Willingness to contribute I'm willing to work on a fix. ### Context Found while reviewing #2906 (predicates on struct-nested leaf columns). That PR fixes the row-filter build path; this binding gap is separate and predates it, but it becomes reachable once predicates on nested leaves work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
