aokolnychyi commented on PR #8278: URL: https://github.com/apache/iceberg/pull/8278#issuecomment-1674052074
Thanks a lot for testing this out, @ConeyLiu! Disabling column stats filtering would clearly be a bad idea in that use case, even though the planning speed improved. I assume that table was partitioned, right? > We also noticed another performance issue here which may be worth mentioning is the decompression for Manifest File. Right now, we can not change the compression codec for the ManifestFile/ManifestListFile. The default one is GZIP. We have implemented some code to support changes by the write.avro.compression-codec. I could submit a PR for it if it is acceptable. Yeah, we use `deflate` by default. It makes sense to have that configurable. I'll need to think a bit whether manifest writers should pick up Avro config for writing data or have its own. Do you want to create a PR, @ConeyLiu? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
