LuciferYang opened a new issue, #18412:
URL: https://github.com/apache/iceberg/issues/18412
### Apache Iceberg version
1.12.0 (latest release)
### Query engine
None
### Please describe the bug 🐞
`ResidualEvaluator` dereferences the partition value in `startsWith` and
`notStartsWith` without a null check, so computing a residual for a partition
whose value is null throws `NullPointerException`. `in` and `notIn` got the
same guard in #17014.
On a table partitioned by `identity(s)`:
```java
table
.newFastAppend()
.appendFile(fileWithPartition("abc"))
.appendFile(fileWithPartition(null))
.commit();
table.newDelete().deleteFromRowFilter(Expressions.startsWith("s",
"a")).commit();
```
```
java.lang.NullPointerException: Cannot invoke "String.startsWith(String)"
because the return value of
"org.apache.iceberg.expressions.BoundReference.eval(org.apache.iceberg.StructLike)"
is null
at
org.apache.iceberg.expressions.ResidualEvaluator$ResidualVisitor.startsWith(ResidualEvaluator.java:215)
```
`ManifestFilterManager` computes a residual for every live file in a
manifest it rewrites, so the delete fails whenever the manifest also holds a
file with a null partition value. Spark's metadata-only delete calls
`deleteFromRowFilter` the same way.
Scans hit it too. With `or(isNull(s), startsWith(s, "a"))`, e.g. `WHERE s IS
NULL OR s LIKE 'a%'`, the null-partition file survives planning through the
`isNull` branch, and `FileScanTask.residual()` throws when the reader asks for
it.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
