richardstartin commented on issue #7973: URL: https://github.com/apache/pinot/issues/7973#issuecomment-1013539250
The problem is you won't get any savings from LZ4 - those CPU readings can be almost identical but with a little bit of noise the data is difficult for a text oriented algorithm like LZ4 to compress. The XOR of any two adjacent values will typically have very few set bits so can result in high compression ratios, perhaps even 8x. Implementing codecs such as xor or delta encoding is a feature that has been discussed before, would not be very difficult, and it would solve your problem in a way making metric columns compressible would not. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
