brgr-s commented on PR #2887: URL: https://github.com/apache/iceberg-rust/pull/2887#issuecomment-5164373346
Just chiming in here that I'd be really happy when this gets merged. I am currently testing PR #2620 as a base for writing compaction in rust, and only just noticed that my implementation using `ParquetWriter::new` exploded my compacted table "byte-wise" and regarding number of files: ``` case input (data files) output (data files) ratio S1 1.47 GB / 150 files 14.35 GB / 19 files x9.76 S2 14.70 GB / 1500 files 143.48 GB / 191 files x9.76 ``` Looking for a solution I realized I could not use `from_table_properties` because specifically `parquet_compression*` and `parquet_row_group_size_bytes` are not wired in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
