moomindani commented on code in PR #3474:
URL: https://github.com/apache/iceberg-python/pull/3474#discussion_r3386942719
##########
pyiceberg/table/puffin.py:
##########
@@ -114,3 +151,92 @@ def __init__(self, puffin: bytes) -> None:
def to_vector(self) -> dict[str, "pa.ChunkedArray"]:
return {path: _bitmaps_to_chunked_array(bitmaps) for path, bitmaps in
self._deletion_vectors.items()}
+
+
+class PuffinWriter:
+ """Writes a Puffin file containing a single deletion-vector-v1 blob."""
Review Comment:
You are right, it didn't. Addressed in eb814227 together with the suggestion
below: PuffinWriter now accepts an OutputFile and finish() writes the file, so
the docstring matches the behavior now.
##########
pyiceberg/table/puffin.py:
##########
@@ -114,3 +151,92 @@ def __init__(self, puffin: bytes) -> None:
def to_vector(self) -> dict[str, "pa.ChunkedArray"]:
return {path: _bitmaps_to_chunked_array(bitmaps) for path, bitmaps in
self._deletion_vectors.items()}
+
+
+class PuffinWriter:
+ """Writes a Puffin file containing a single deletion-vector-v1 blob."""
+
+ _blobs: list[PuffinBlobMetadata]
+ _blob_payloads: list[bytes]
+ _created_by: str
+
+ def __init__(self, created_by: str | None = None) -> None:
Review Comment:
Good idea, done in eb814227. PuffinWriter now takes an OutputFile and
finish() writes the content to it and returns the file size, following the Java
PuffinWriter shape. One simplification compared to Java: the file is assembled
in memory and written in one shot rather than streamed, which should be fine
for DVs since they are small. Happy to revisit with a streaming implementation
if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
