Yukang-Lian opened a new pull request, #48759:
URL: https://github.com/apache/doris/pull/48759

   ### What problem does this PR solve?
   
   Issue Number: close #xxx
   
   Related PR: #xxx
   
   Problem Summary:
   
   This PR introduces a delay strategy for cumulative compaction tasks that 
protects the thread pool from becoming overwhelmed by resource-intensive 
operations.
   
   The new `should_delay_submission()` function implements a delay strategy 
that:
   - Only activates when the thread pool size meets a configurable minimum 
threshold
   - Checks if the thread pool is near capacity (0-1 threads remaining)
   - Evaluates whether the compaction task is resource-intensive based on input 
size and row count
   
   When these conditions are met, the function returns true to indicate the 
submission should be delayed, preventing potential thread pool exhaustion and 
system performance degradation.
   
   The implementation includes three configurable thresholds:
   - `min_threads_for_cumu_delay_strategy`: Minimum thread pool size to 
activate the strategy
   - `cumu_delay_strategy_size`: Maximum input size threshold (in bytes)
   - `cumu_delay_strategy_row_num`: Maximum input row count threshold
   
   Appropriate warning logs are generated when a resource-intensive task is 
delayed, including the tablet ID and resource metrics for easier 
troubleshooting.
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: 
https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should 
merge into -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Reply via email to