badalprasadsingh opened a new issue, #2605: URL: https://github.com/apache/iceberg-go/issues/2605
### Apache Iceberg version main (development) ### Please describe the bug 🐞 after the source column of an identity partition is promoted (`int` to `long`, or `float` to `double`), a v2 merge-on-read `Delete` commits fine, but every scan after it fails: ``` invalid metadata: position delete file .../data/v=1.5/00000-0-857e...-00001.parquet partition does not match data file .../data/v=1.5/00000-0-04a5...-00001.parquet ``` what I mean: with `write.delete.mode=merge-on-read`: 1. v2 table `id long, v float`, partitioned by `identity(v)` 2. append `(1, 1.5)` and `(2, 1.5)` 3. promote `v` to `double` with `UpdateSchema().UpdateColumn` 4. `Delete(v = 1.5 AND id = 1)`: commits 5. `Scan()`: fails with the error above `int` to `long` fails the same way. v3 tables (deletion vectors) are not affected. both files are in the same partition. The data file's manifest was written before the promotion, so its partition value is read back as a `float32` (`int32`). The delete file's manifest was written after it, so its value is a `float64` (`int64`) -> the check during (`appendPositionalDeletesFromSequence` calling `partitionsMatch`) compares the two Go values as they are, so equal values never match. In v0.7.0 the same `Delete` fails (`invalid binary serialization: expected 8 bytes for float64 value, got 4`), so nothing is committed. On main the delete commits, and after that go can no longer scan the table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
