deepthi912 opened a new pull request, #18020: URL: https://github.com/apache/pinot/pull/18020
- Replaced segments and removed segments are not cleaned up from tracking set (`_updatedSegmentsSinceLastSnapshot`): Pinot tracks which segments need their state saved to disk. When a segment gets replaced (e.g. during a reload), the old segment is removed from the active list but not from the tracking set. It stays there until the next periodic save which only happens every 3-20 hours. Each partition leaks ~248 MB this way. - Cloning the bitmap while persisting snapshot of all the segments that are updated would bring up the heap usage to pretty high value if the segment commits happen frequently. Instead of doing so, synchronized the serialization process of existing `ThreadSafeMutableRoaringBitmap` to bring down the number of objects created by this class. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
