rambleraptor commented on code in PR #3474:
URL: https://github.com/apache/iceberg-python/pull/3474#discussion_r3546872001
##########
pyiceberg/table/deletion_vector.py:
##########
@@ -67,6 +93,24 @@ def _deserialize_bitmap(pl: bytes) -> list[BitMap]:
return bitmaps
+ @staticmethod
+ def _serialize_bitmap(bitmaps: list[BitMap]) -> bytes:
+ # Counterpart of _deserialize_bitmap: number of bitmaps (8 bytes,
little-endian), then for each
+ # non-empty bitmap in ascending key order its key (4 bytes,
little-endian) and serialized payload.
+ non_empty = [(key, bitmap) for key, bitmap in enumerate(bitmaps) if
len(bitmap) > 0]
+
+ with io.BytesIO() as out:
+ out.write(len(non_empty).to_bytes(8, "little"))
+ for key, bitmap in non_empty:
+ if key > MAX_JAVA_SIGNED:
Review Comment:
I think we've got an off-by-one error here.
Here's
[Java](https://github.com/apache/iceberg/blob/20bf72be73e9b4aa1dec30913e891fe8adabd8c5/core/src/main/java/org/apache/iceberg/deletes/RoaringPositionBitmap.java#L305)
Java fails if key > (2^31 - 1) - 1
We fail if key > (2^31 ) - 1
This is a weird edge case and definitely an issue in Java, but probably not
worth fixing over there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
