mcvsubbu commented on issue #6556: URL: https://github.com/apache/incubator-pinot/issues/6556#issuecomment-775346288
Thanks, @vmarchaud . Does pubsub partition the stream, or is it all in one partition? If there is only one partition, then that is another reason why it may not work in some production use cases with very high ingestion rate. We will be limited by the consumption capacity of a single server to consume the complete stream (this is another reason we moved away from HLC). On the other hand, a partitioned stream allows us to spread the consumption across multiple servers, thus allowing us to scale horizontally as ingestion rate increases. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
