mbutrovich commented on code in PR #3302:
URL: https://github.com/apache/iceberg-rust/pull/3302#discussion_r4188555829
##########
crates/iceberg/src/arrow/reader/pipeline.rs:
##########
@@ -603,6 +604,14 @@ impl FileScanTaskReader {
&predicate,
use_position_fallback,
)?;
+ let predicate = residual_for_missing_fields(
Review Comment:
I added both versions. `test_residual_for_missing_fields` now has the two
keep predicates. With `b` defaulting to 7, `b IS NULL OR b != 7` becomes `false
OR false` and `b IS NULL OR b != 5` becomes `false OR true`. The residual keeps
the `OR` node and doesn't fold it into one constant.
`assert_absent_b_reads_as_7` also reads the file with an [equality delete
file](https://github.com/apache/iceberg/blob/5e7169168db3d34e29354c6f59ec4d6e420b8d2d/format/spec.md?plain=1#L1417-L1425)
(`equality_ids: Some(vec![2])`) that deletes `b` = 7, `b` = 5, or `b` = null,
and checks that 0, 3, and 3 rows remain. Both reader tests go through that
helper, so the deletes are checked with `b` coming from `initial-default` and
from an identity partition value. With the residual call removed from
`pipeline.rs`, the delete on 7 keeps all 3 rows, which matches your trace.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
