syun64 commented on code in PR #358:
URL: https://github.com/apache/iceberg-python/pull/358#discussion_r1477113742
##########
pyiceberg/io/pyarrow.py:
##########
@@ -1720,13 +1720,22 @@ def write_file(table: Table, tasks:
Iterator[WriteTask]) -> Iterator[DataFile]:
except StopIteration:
pass
+ compression_codec = table.properties.get("write.parquet.compression-codec")
+ compression_level = table.properties.get("write.parquet.compression-level")
+ if compression_codec == "uncompressed":
+ compression_options = {"compression": "none"}
+ else:
Review Comment:
How do you feel about checking if `compression_codec is not None` before
adding the compression option to the ParquetWriter kwarg?
I think we can use the meaningful distinction between None and string 'NONE'
to mean either that the compression codec was not set (and hence should use the
default) versus 'NONE' meaning that we'd like to avoid compression altogether.
I think this might help clean up some of the if conditions in
**test_parquet_compression**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
