hedger9487 opened a new pull request, #3842: URL: https://github.com/apache/iceberg-python/pull/3842
### Description Fixes #3840. In the Iceberg specification (`format/spec.md`), Java reference implementation, and `iceberg-cpp`, `equality_ids` (field 135) is defined as a list of `int` (`list<136: int>`). However, `pyiceberg` previously declared field 136 with `LongType()`, causing manifests written by PyIceberg to be rejected by other spec-conformant readers (e.g. `iceberg-cpp`). This PR: 1. Updates `DATA_FILE_TYPE` in `pyiceberg/manifest.py` for table format versions 1, 2, and 3 to use `IntegerType()` for element id 136 in `equality_ids`. 2. Updates `ReadSchemaResolver.primitive` in `pyiceberg/avro/resolver.py` to allow promoting `LongType` in file schema to `IntegerType` in read schema for Avro decoding (since both are encoded as zigzag varints on the wire), ensuring backward compatibility when reading historical manifests written by earlier versions of PyIceberg. 3. Adds unit tests verifying that newly written manifests serialize `equality_ids` with element type `int`, and that historical manifests with element type `long` can be read seamlessly. ### Testing - Added unit tests in `tests/utils/test_manifest.py`. - All 42 manifest tests and full test suites pass locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
