zhang-arvin commented on issue #17301:
URL: https://github.com/apache/iceberg/issues/17301#issuecomment-5836317097
This appears to be already fixed on `main` — posting the evidence so it can
be closed rather than picked up.
The crash is real for the reported version. At `1ec15051` (and 1.11.0)
`notNaN` read the nullability map *before* the absent-column guard, so
`mayContainNulls.get(id)` unboxed a `null` for a column missing from the file:
```java
if (mayContainNulls.get(id)) { // NPE on a missing column
return ROWS_MIGHT_MATCH;
}
Boolean hasNonDictPage = isFallback.get(id); // guard came after
```
**But #17633 ("Parquet: Fix notNaN for columns missing from files", merged
2026-08-13 by @RussellSpitzer) swapped those two blocks and added
`notNaN("not_in_file")` to `testColumnNotInFile`.** That is byte-for-byte the
same change as the never-merged #17302, and it landed against #17632. Current
`main` therefore has the guard first:
```java
Boolean hasNonDictPage = isFallback.get(id);
if (hasNonDictPage == null || hasNonDictPage) {
return ROWS_MIGHT_MATCH; // missing column returns
here
}
if (mayContainNulls.get(id)) { // no longer reachable with
a null
return ROWS_MIGHT_MATCH;
}
```
Verified against `origin/main` at `212aa894e`: the repro no longer fails
(`id` for `not_in_file` has `isFallback == null` and returns `ROWS_MIGHT_MATCH`
before the nullability lookup is reached), and `TestDictionaryRowGroupFilter`
passes on a clean tree — 76 tests, 0 failures.
The other unboxing sites in this class (`notEq`, `notIn`, `notStartsWith`)
are not affected either: `isFallback` and `mayContainNulls` are populated
together for the same `id`, and each of those methods already checks
`isFallback` first, so `mayContainNulls.get(id)` cannot be `null` there.
Suggest closing this as completed. Note #17633 references #17632 rather than
this issue, which is why a per-issue search for a fix here comes up empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
