blackmwk commented on code in PR #2887:
URL: https://github.com/apache/iceberg-rust/pull/2887#discussion_r3710967356
##########
crates/iceberg/src/writer/file_writer/parquet_writer.rs:
##########
@@ -46,6 +47,16 @@ use crate::transform::create_transform_function;
use crate::writer::{CurrentFileStatus, DataFile};
use crate::{Error, ErrorKind, Result};
+/// Default compression levels, pinned to parquet-java's defaults so files are
+/// comparable across implementations rather than tracking the parquet-rs
+/// defaults, which may differ and could change without us noticing.
+/// Default zstd level (parquet-rs uses 1).
+const DEFAULT_ZSTD_COMPRESSION_LEVEL: i32 = 3;
+/// Default gzip level.
+const DEFAULT_GZIP_COMPRESSION_LEVEL: u32 = 6;
+/// Default brotli level.
+const DEFAULT_BROTLI_COMPRESSION_LEVEL: u32 = 1;
Review Comment:
I don't think we should maintain them here, they are part of compression
module.
##########
crates/iceberg/src/spec/table_properties.rs:
##########
@@ -168,6 +190,20 @@ pub struct TableProperties {
pub cdc_max_chunk_size: usize,
/// Content-defined chunking normalization level (gearhash bit adjustment).
pub cdc_norm_level: i32,
+ /// Parquet compression codec name (e.g. `zstd`, `gzip`). Validated when
the
+ /// writer is built, not when properties are parsed.
+ pub parquet_compression_codec: String,
Review Comment:
We should use `CompressionCodec`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
