kangpinghuang opened a new issue #2712: [SegmentV2] optimize fd management in Segment V2 URL: https://github.com/apache/incubator-doris/issues/2712 In a situation with many load jobs, there are risks that BE will core dump because of Too Many Open File error. The reason is that in segment v2, the Segment will hold fd before the rowset is closed. When there are many load jobs, the compacted rowset can not be deleted for incremental rowsets will hold them for 30min, although compactions are quick enough. So we should add a mechanism to close the fd when we finished read file and add a fd cache for performance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
