freemandealer opened a new pull request, #70143:
URL: https://github.com/apache/doris/pull/70143
### What problem does this PR solve?
Issue Number: None
Related PR: #60238
Problem Summary:
Transient task submission checks ring-buffer capacity separately from
publication. Concurrent submissions can still wait for capacity, and
publication can race with closing admission. A failed publication can also
leave an executor registered, especially when it throws an unchecked exception;
duplicate task IDs can replace an active executor.
Use the existing multi-producer ring buffer's atomic `tryPublishEvent` under
a short admission lock shared with `close()`. Reject full, unstarted, or closed
schedulers explicitly, and drain accepted work outside the lock. Register
executors with `putIfAbsent`, and undo only the matching registration on
publication failure while preserving its cause as a `JobException`. Remove
unused publication overloads that bypass transient-task admission.
Add deterministic coverage for queue saturation, concurrent wrap-around,
publication/close ordering, duplicate IDs, failure cleanup, and the Export and
connector rewrite callers. The rewrite tests target the current
`ConnectorRewriteDriver` and retain its drain-before-rollback behavior.
Validation on master `7866f5a36c02e15dc014aaef30e537a395f4059d`:
- Full FE build and packaging with `build.sh --fe -j16`, including UI assets
built from the committed lockfile with Node 22/npm 10.
- Explicit Checkstyle for `fe-common` and `fe-core`, including test sources:
zero violations.
- `run-fe-ut.sh --run` for `TaskDisruptorTest`, `TransientTaskManagerTest`,
`ExportMgrTest`, `ConnectorRewriteDriverTest`, and
`ConnectorRewriteGroupTaskTest`: 38 tests, zero failures, errors, or skips.
- `git diff --check`.
### Release note
Fix FE transient-task submission so queue saturation rejects without waiting
for capacity, publication is ordered with scheduler closure, and failed or
duplicate submissions do not leave or overwrite executor registrations.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test.
- Behavior changed:
- [ ] No.
- [x] Yes. Full, unstarted, closed, or duplicate submissions fail
explicitly, and publication failure cleans up its own registration.
- Does this need documentation?
- [x] No. No new configuration or SQL syntax; the existing queue
configuration comment is updated.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
