mcvsubbu commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1399174396
> Agree with the advantages of segment size based threshold, but it also has some drawbacks: > > 1. User sometimes wants to fix the row count per segment. It is not straight forward to translate segment size into row count, and it also differs with different data distribution. > 2. When moving to a new server, we need to start with a small segment, then slowly increase the row count to get the desired segment size. that is not true. Only on a new table we start with a small segment and increase. New servers/rebalance is not a problem. > > Even for segment size based threshold, we translate the desired segment size into row count. IMO we should give user the option to directly configure the segment row count. For a lot of use cases, memory is not really a concern, and I myself find it difficult to tune the segment size, not mention the average user. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
