siddharthteotia opened a new pull request #5511: URL: https://github.com/apache/incubator-pinot/pull/5511
## Description **Change is WIP**. We should generate all indexes in LITTLE-ENDIAN format considering that Pinot deployments are on x86. Right now all indexes are BIG-ENDIAN and this results in byte swapping (per read on the buffer) during query processing on the server. While this provides compatibility with machines that are generating segments outside Pinot, but it is not needed unless someone is either generating segments on Solaris/SPARC/PowerPC etc or running Pinot on one of these architectures. With deployments on x86, we should be able to use LITTLE-ENDIAN. The only advantage of BIG-ENDIAN is that it is **standard network byte order** so the socket won't swap bytes for downloading the BIG-ENDIAN segments. But I think it is fine to tolerate the overhead once during download at the socket layer as opposed to during every read operation during query execution. This PR doesn't change any format. It is just putting together skeleton code that is needed to make our code flexible with byte order format as there are assumptions everywhere that each index file is big endian. Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion) * [ ] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes) Does this PR fix a zero-downtime upgrade introduced earlier? * [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes) Does this PR otherwise need attention when creating release notes? Things to consider: - New configuration options - Deprecation of configurations - Signature changes to public methods/interfaces - New plugins added or old plugins removed * [ ] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes) ## Release Notes If you have tagged this as either backward-incompat or release-notes, you MUST add text here that you would like to see appear in release notes of the next release. If you have a series of commits adding or enabling a feature, then add this section only in final commit that marks the feature completed. Refer to earlier release notes to see examples of text ## Documentation If you have introduced a new feature or configuration, please add it to the documentation as well. See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
