itschrispeck opened a new pull request, #15685: URL: https://github.com/apache/pinot/pull/15685
Add `maxBytesSize` config to limit mutable json index heap usage. When data is unpredictable, we often see JSON index can spike to use a substantial amount of heap. This PR adds a config to limit the amount of data in the mutable json index. Due to the current structure of the on-disk index, this happens to be strong indicator of disk size too, and can help prevent sporadic large segments. Also adds a metric, emitted on index close, for the heap memory used by the key and key+value strings. It is useful to track expensive tables, and difficult to quickly aggregate this info from logs. Default limit is Long.MAX, so effectively no change to existing behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
