rajendrag opened a new issue, #18672:
URL: https://github.com/apache/iceberg/issues/18672
### Apache Iceberg version
1.10.0
### Query engine
Spark
### Please describe the bug 🐞
For a table partitioned by a nullable date column, a metadata DELETE using
`<` can incorrectly remove rows from the NULL partition. The behavior depends
on the manifest layout.
For example, inserting the rows together removes the NULL row, while
inserting them separately preserves it.
```
CREATE TABLE t_one (id int, d date)
USING iceberg PARTITIONED BY (d)
TBLPROPERTIES ('format-version'='2', 'write.delete.mode'='merge-on-read');
INSERT INTO t_one VALUES
(1, DATE '2026-10-01'),
(2, CAST(NULL AS DATE)),
(3, DATE '2026-10-20');
DELETE FROM t_one WHERE d < DATE '2026-10-10';
SELECT id FROM t_one ORDER BY id;
-- Expected: 2, 3. Actual: 3.
CREATE TABLE t_sep (id int, d date)
USING iceberg PARTITIONED BY (d)
TBLPROPERTIES ('format-version'='2', 'write.delete.mode'='merge-on-read');
INSERT INTO t_sep VALUES (1, DATE '2026-10-01');
INSERT INTO t_sep VALUES (2, CAST(NULL AS DATE));
INSERT INTO t_sep VALUES (3, DATE '2026-10-20');
DELETE FROM t_sep WHERE d < DATE '2026-10-10';
SELECT id FROM t_sep ORDER BY id;
-- Expected and actual: 2, 3.
```
Local core API tests confirm the manifest dependency: the NULL-partition
file is incorrectly deleted when it shares a manifest with a matching file, but
survives in a separate manifest. `ResidualEvaluator` uses a comparator that
sorts NULL before non-NULL values, incorrectly simplifying `<` and `<=` to true
for NULL partitions.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
