linzch3 opened a new pull request, #3622: URL: https://github.com/apache/iceberg-rust/pull/3622
## Which issue does this PR close? - Closes #3618. ## What changes are included in this PR? `get_parquet_stat_min_as_datum` and `get_parquet_stat_max_as_datum` now return no bound for `string`, `binary`, `fixed` and `uuid` when the Parquet statistics are the deprecated `min`/`max` fields (`is_min_max_deprecated()`) and `min != max`. parquet-mr before 1.10 wrote those with signed byte order, so the row-group evaluator could skip groups with matching rows (e.g. `é` sorting before `a`). This matches parquet-java's `ParquetMetadataConverter`, which keeps deprecated bounds only for signed-sort types or when `min == max`. Signed-sort types such as decimals are unchanged. ## Are these changes tested? Yes, with unit tests: - `test_deprecated_unsigned_parquet_statistics` (schema.rs): deprecated `é`/`a` bounds give `None`, deprecated `min == max` and non-deprecated bounds are kept. - `eval_true_for_deprecated_string_bounds_filter_eq` (row-group evaluator): the issue's case, `s = 'é'` with deprecated bounds `é`/`a`, now keeps the row group. Both fail without the fix. I ran `cargo test -p iceberg --lib`, `cargo clippy -p iceberg --all-targets -- -D warnings` and `cargo fmt` locally. ## AI Disclosure This change was prepared with an AI coding assistant (Grok Bot); the tests above were run locally. Areas of uncertainty: the set of unsigned-sort types is taken from the Parquet spec and parquet-java rather than tested against real pre-1.10 files. The same helper also affects `parquet_writer`'s stats collection, where it means no bound is recorded for such columns, which is the conservative choice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
