geekboood commented on issue #10960: URL: https://github.com/apache/pinot/issues/10960#issuecomment-1634674328
@Jackie-Jiang I tried to figure out the memory leak by using pmap to investigate the memory layout. I found that it allocated a lot of anonymous memory, each with 6x MB. After using gdb to dump these memory regions, I found my data for a column. While reading through the code, I discovered that ColumnValueReader implements the Closeable interface, but DataFetcher put it into a hashmap. Will it automatically call the close method sometime? If no one invokes these close methods, the direct memory allocated by ChunkReaderContext will not be freed. These are used in VarByteChunkSVForwardIndexReader to decompress string column data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
