morningman opened a new issue #6701: URL: https://github.com/apache/incubator-doris/issues/6701
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version trunk fee8e6a ### What's Wrong? BE OOM and crash after loading a large volume of data. For example, there is only 1 tablet in a table, and if we loading 100GB data in one broker load job. After loading finished, the OOM will happen. This is because there are too many segments in one rowset, so that when doing compaction for this rowset, it will take too much memory, which cause OOM. ### What You Expected? Don't crash. ### How to Reproduce? Loading a large volume of data in one load job to a table with a few tablets. ### Anything Else? I am about to limit the number of segments in the newly created segment. If the number of segment exceed limit, load will failed with error code -238. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
