mcvsubbu commented on issue #8804: URL: https://github.com/apache/pinot/issues/8804#issuecomment-1155785311
Here is the history: When we had only HLC, we had `replication` for both OFFLINE as well as REALTIME tables. In case of REALTIME, it was used to [derive](https://github.com/apache/pinot/blob/master/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotTableIdealStateBuilder.java#L89) the number of realtime servers to consume the topic and also whether the topic was divided amongst the servers (something that only Uber used, I believe -- to split events on same topic across servers). Therefore, this config could not be used for LLC, and we picked a new config -- `replicasPerPartition`. Note that `replication` is still used for OFFLINE tables. I am not sure if we can declare HLC dead as yet. We can certainly poll the community and check, and if everyone agrees, we can remove the code, and proceed to treat `replication` as the number of replicas -- whether offline or realtime. Another way to do this would be to use `replication` as the number of replicas if a table has only LLC (but then we have to make sure that they never ever add HLC to it -- having both HLC and LLC is one way of migrating from HLC to LLC). Fall back to `replicasPerPartition` if that `replicas` is not defined. This may be a more palatable way in case people are using HLC. Happy to discuss over a small doc rather than this issue. We can then float the doc to the public as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
