richardstartin commented on pull request #7797: URL: https://github.com/apache/pinot/pull/7797#issuecomment-974519284
> LZ4 gives nearly same (and even better sometimes) the compression ratio compared to SNAPPY and the time to compress is much faster. > ZSTD on the other hand as highest compression ratio (from our tests) but compression/decompression time was higher as well This is consistent with my experience, but it’s worth noting LZ4 has 16 compression levels which trade speed for ratio, and we default to level 1, whereas beyond level 3 a high compression mode is used - if ratio is really important the level could be exposed as a config parameter. But right now the motivation is faster decompression by default, rather than smaller file sizes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
