LuciferYang opened a new pull request, #18414:
URL: https://github.com/apache/iceberg/pull/18414
Closes #18412.
`ResidualVisitor.startsWith` and `notStartsWith` call `startsWith` on the
partition value without a null check, so the residual for a null partition
throws `NullPointerException`. `ManifestFilterManager` computes a residual for
each live file in a manifest it rewrites, so
`deleteFromRowFilter(startsWith("s", "a"))` on an `identity(s)` table fails
whenever that manifest also holds a file with a null partition value. Scans hit
it with filters like `or(isNull(s), startsWith(s, "a"))`: the null-partition
task survives planning through `isNull`, and `FileScanTask.residual()` throws.
This returns `alwaysFalse` for `startsWith` and `alwaysTrue` for
`notStartsWith` when the value is null, the same as `in`/`notIn` (#17014) and
the row-level `Evaluator`. One consequence:
`deleteFromRowFilter(notStartsWith("s", "a"))` now deletes null-partition files
instead of failing, which is what `notEqual` and `notIn` already do.
New tests:
- `TestResiduals`: null partition for `startsWith` and `notStartsWith`, and
the `or(isNull, startsWith)` shape.
- `TestTruncatesResiduals`: null partition under `truncate`.
- `TestDeleteFiles.testDeleteByStartsWithKeepsNullPartition`: the delete
above keeps the null-partition file.
---
**AI Disclosure**
- Model: Claude Opus
- Human Oversight: reviewed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
