Fokko commented on code in PR #4395: URL: https://github.com/apache/iceberg-python/pull/4395#discussion_r4600173214
########## pyiceberg/avro/codecs/zstandard_codec.py: ########## @@ -16,38 +16,53 @@ # under the License. from __future__ import annotations +import sys from io import BytesIO from pyiceberg.avro.codecs.codec import Codec -try: - from zstandard import ZstdCompressor, ZstdDecompressor +if sys.version_info >= (3, 14): + from compression import zstd Review Comment: PEP 784 makes `compression.zstd` optional; it's only built when libzstd is available (e.g. pyenv building 3.14 on a machine without the libzstd headers won't have it). On such a build this import raises at module load, and since `pyiceberg/avro/codecs/__init__.py` imports `ZStandardCodec` eagerly, `import pyiceberg.manifest` fails. That means reading *any* manifest breaks, not just zstd-compressed ones. Today the `try/except` defers the error until zstd is actually used. On top of that, with the marker in `pyproject.toml` users can't recover by installing `zstandard`, since this branch never tries it. Could we fall back instead: try `compression.zstd`, then `zstandard`, then the stub that raises on use? The same applies to `puffin.py`. It might be cleanest to put this in one small module that both use, so the import switch lives in one place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
