vishnuprakaz opened a new pull request, #3523:
URL: https://github.com/apache/iceberg-python/pull/3523
Closes #3522
# Rationale for this change
`bytes_required` should return the *minimum* number of bytes for
a value, but it returns one byte too many for negatives like
-128 and -32768.
This matters because the decimal bucket transform hashes those
bytes. The extra byte changes the hash, so PyIceberg can put a
value in a different bucket than Spark/Java. For example,
`Decimal("-1.28")` goes to bucket 12 in PyIceberg but bucket 13
everywhere else.
The fix computes the length from `(value + 1)` for negative
values, which gives the true minimum and matches the Iceberg
spec.
## Are these changes tested?
Yes. Added tests for the -128 / -32768 / -8388608 boundary cases
(where the old code was wrong) plus positive/negative controls.
Lint and the decimal/conversion/transform tests pass.
## Are there any user-facing changes?
Yes. For decimal values like -1.28, the computed bucket changes
(e.g. 12 → 13) so it now matches other Iceberg engines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
