yixiutt opened a new pull request, #10153: URL: https://github.com/apache/incubator-doris/pull/10153
add some logic to opt compaction: 1.seperate base&cumu compaction in case base compaction runs too long and affect cumu compaction 2.fix level size in cu compaction so that file size below 64M have a right level size, when choose rowsets to do compaction, the policy will ignore big rowset, this will reduce about 25% cpu in high frequency concurrent load 3.remove skip window restriction so rowset can do compaction right after generated, cause we'll not delete rowset after compaction. This will highly reduce compaction score in concurrent log. 4.remove version consistence check in can_do_compaction, we'll choose a consecutive rowset to do compaction, so this logic is useless after add logic above, compaction score and cpu cost will have a substantial optimize in concurrent load. # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
