ankitsultana commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1398771142
New config since we wouldn't want to break the existing behavior. The existing behavior is by design as @mcvsubbu had mentioned. A note on the motivation for this issue: I agree with the point about being careful about adding new configs, but what we have seen internally is that almost every time what we are interested in is setting a threshold on the exact number of rows per segment. We have come close to incidents a few times when dealing with high ingestion throughput tables because someone set a flush threshold thinking it will directly control the number of rows, but it ended up dividing the number of rows by maxPartitionsPerInstance, leading to very frequent flushes which put pressure on controller/deep-store. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
