mcvsubbu commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396307559
> @mcvsubbu The proposed way is very useful when we want to limit the CPU usage for all the consumers on a given server, but for certain use cases, CPU might not be the bottleneck (e.g. ingestion rate is low, query throughput is low, no strict latency requirement), and we just want an easy way to configure the segment size so that segment is not too small or too big. It will be similar to the size (in bytes) based config, but we fix the number of rows per segment. It will be easier to config and get desired performance than the size based because for wide or narrow table, size based will give very different row count per segment. For columnar store, row count is the major factor of the query performance. Also, the size based config can only be properly set after the first several segments are consumed, which is not as straight-forward as the row based. I don't agree with that last part. The size is recommended by the recommendation engine, and can be configured in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
