Fokko commented on code in PR #4395:
URL: https://github.com/apache/iceberg-python/pull/4395#discussion_r4600173214


##########
pyiceberg/avro/codecs/zstandard_codec.py:
##########
@@ -16,38 +16,53 @@
 # under the License.
 from __future__ import annotations
 
+import sys
 from io import BytesIO
 
 from pyiceberg.avro.codecs.codec import Codec
 
-try:
-    from zstandard import ZstdCompressor, ZstdDecompressor
+if sys.version_info >= (3, 14):
+    from compression import zstd

Review Comment:
   PEP 784 makes `compression.zstd` optional; it's only built when libzstd is 
available (e.g. pyenv building 3.14 on a machine without the libzstd headers 
won't have it). On such a build this import raises at module load, and since 
`pyiceberg/avro/codecs/__init__.py` imports `ZStandardCodec` eagerly, `import 
pyiceberg.manifest` fails. That means reading *any* manifest breaks, not just 
zstd-compressed ones. Today the `try/except` defers the error until zstd is 
actually used.
   
   On top of that, with the marker in `pyproject.toml` users can't recover by 
installing `zstandard`, since this branch never tries it.
   
   Could we fall back instead: try `compression.zstd`, then `zstandard`, then 
the stub that raises on use? The same applies to `puffin.py`. It might be 
cleanest to put this in one small module that both use, so the import switch 
lives in one place.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to