chenboat opened a new pull request, #16432: URL: https://github.com/apache/pinot/pull/16432
Add a configuration option to skip segment CRC check when load a segment during server start or pre-downloading. The default value is still true to check segment CRC so there is no back compatibility issues. As Pinot segment today include many variety of indexes and data, it become increasingly hard to use CRC to verify segment consistency across replicas. Examples include Lucene text index which has by design non-deterministic behavior during indexing. There should be a revamp on how to use and create CRC when loading segments. This PR just creates option for users to opt out. It has been tested in our production env for a while for Pinot table with text index without issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
