sollhui opened a new pull request, #57133:
URL: https://github.com/apache/doris/pull/57133
### What problem does this PR solve?
This PR implements a backpressure algorithm to control load speed when
tablets accumulate too many versions, preventing `-235` errors.
Key changes:
- Add configuration `max_load_pressure_wait_time_ms` to control maximum
sleep time
- Implement backpressure algorithm in VNodeChannel with sleep mechanism
based on tablet version count
- Add `set_tablet_load_rowset_num_info` method to BaseDeltaWriter for load
pressure info collection
- Support both traditional and cloud tablet implementations:
- Traditional: uses `tablet_meta()->version_count()` and
`config::max_tablet_version_num`
- Cloud: uses `cloud_tablet()->fetch_add_approximate_num_rowsets(0)` and
`cloud_tablet()->max_version_config()`
- Update LoadChannel, TabletsChannel, and InternalService to propagate load
pressure info
- Add load pressure time tracking to WriterStats for monitoring
The algorithm works by:
1. Collecting tablet version count info from DeltaWriters
2. Calculating sleep time based on version count gap when tablets exceed 50%
capacity
3. Applying backpressure sleep in VNodeChannel::try_send_and_fetch_status to
slow down data loading
4. Reporting total load pressure time for performance monitoring
This helps prevent version accumulation and improves stability during
high-load scenarios.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
