mcvsubbu commented on issue #6121: URL: https://github.com/apache/incubator-pinot/issues/6121#issuecomment-706720810
Pinot uses heap memory only for query execution (and then some one off operations like building indexes during load, etc.) In hosts that have consuming segments, heap memory is also use fo inverted indices, and segment build upon completion. Inverted indices for the consuming segment may expand as rows are ingested, and released in one go when the consuming segment completes. We have seen gc overload at that time. Everything else uses offheap memory -- either mmapped or direct. At linkedin, we typically have xmx of 16G, and use mmap for segments, and realime consumers. Coming back to your use case, how many replicas do you have? (we need to output that as well, on the command line. I will fix that). What was the value of xmx you provided? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
