YinZheng-Sun opened a new pull request, #992: URL: https://github.com/apache/iceberg-cpp/pull/992
Parquet schema projection currently accepts only Arrow Decimal128 for an Iceberg decimal. Arrow can also return Decimal32/64 when `smallest_decimal_enabled` is enabled or when those types are restored from stored `ARROW:schema` metadata (apache/arrow#47427). The latter also affects the Iceberg reader with default reader properties: a valid file fails with `Cannot read Iceberg type: decimal(20, 2) from Parquet type: decimal32(9, 2)`. Accept Decimal32/64 alongside Decimal128 and use the common Arrow `DecimalType` interface for the existing precision and scale checks. Precision widening remains allowed; narrowing and scale changes remain rejected. The default inference of ordinary Parquet DECIMAL columns remains Decimal128. Regression coverage includes: - INT32/INT64-backed decimal schema projection with `smallest_decimal_enabled` both enabled and disabled, checking equal/wider precision and rejecting narrower precision or a different scale. - Reading actual Parquet files with stored Decimal32/64 Arrow schemas through the Iceberg reader, verifying negative, zero, and positive values after promotion to Decimal128. Validation on macOS with Arrow 25.0.1: - Both new regression tests fail before the fix and pass after it. - `ctest --test-dir build -R '^parquet_test$' --output-on-failure -V`: 112 passed, 1 skipped because all optional compression codecs are available. - `uvx --from pre-commit==4.5.1 pre-commit run --all-files`: passed. AI assistance: Codex helped draft the fix and regression tests and ran the local verification above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
