0AyanamiRei opened a new pull request, #68006:
URL: https://github.com/apache/doris/pull/68006
### What problem does this PR solve?
Related PR: #62661
Problem Summary:
Cloud random distribution tables turn adaptive random bucket mode on for the
sink
(`enable_adaptive_random_bucket_load`, default true), and the FE is expected
to compute, for every
sink backend, the bucket owner backend and the receiver local bucket
sequence through
`OlapTableSink.computeAdaptiveRandomBucketAssignments` /
`applyAdaptiveRandomBucketAssignments`.
Only `Coordinator`, `ThriftPlansBuilder` and the runtime partition
creation/replace paths do that.
The stream load planning path (`NereidsStreamLoadPlanner` ->
`StreamLoadHandler`) never did, so the
BE received an adaptive sink without any assignment, fell back to treating
the executing BE as the
owner of every bucket (`vtablet_writer.cpp: adaptive_bucket_be_id`) and
failed with
```
unknown partition channel, load_id=..., index_id=..., partition_id=...
```
as soon as a partition has no tablet on that BE. For a `DISTRIBUTED BY
RANDOM BUCKETS 1` table this
happens for every redirect that does not pick the single owning backend;
with more backends than
buckets the load fails whenever the entry backend owns none of the
partition's tablets.
Observed on a cloud regression run: 8 stream loads of single bucket random
tables failed with the
error above, while the same suite passed whenever the entry backend happened
to be the tablet
owner.
### What this PR changes
* Resolve the adaptive routing for the backend that issued the request
(`TStreamLoadPutRequest.backendId`, which the BE always sets for its own
stream load) right before
the plan is returned, reusing the existing compute/apply helpers. The sink
then routes each
partition to a backend it can actually reach, and the receiver side gets a
matching local bucket
sequence.
* When the request carries no usable backend id (old clients, multi table
stream load), no
assignment consistent with the receiver side can be computed. In that case
adaptive mode is turned
off for the plan so the BE uses the legacy per-batch tablet routing,
instead of letting it guess
the bucket owner. This keeps the pre-#62661 behavior instead of failing.
Routine load picks the executing BE after the plan is generated, so it is
not covered here and
still needs the sink backend id at planning time;
`load_to_single_tablet=true` remains unaffected.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [x] Yes. <!-- Explain the behavior change -->
Stream load plans of cloud random distribution tables now carry the
adaptive bucket
assignment, and requests without a backend id fall back to the
non-adaptive routing.
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
