aruraghuwanshi opened a new pull request, #20674:
URL: https://github.com/apache/druid/pull/20674
### Description
Operators draining a Middle Manager currently list running tasks, GET each
payload, and POST `taskGroups/handoff` per supervisor with explicit group ids.
There is no Overlord call that queues early handoff for the streaming tasks on
one worker.
This PR adds `POST /druid/indexer/v1/worker/{host}/handoff`. `{host}` is the
same `hostname:port` already used by worker enable/disable. An optional `types`
body filters to `index_kafka` / `index_kinesis`; omit it to include both. The
Overlord groups matching tasks by `supervisorId` + `taskGroupId` and reuses
`SupervisorManager.handoffTaskGroupsEarly`. Success is 202 with the queued map.
Empty workers are 202. Unknown host is 404. A `types` list with no
early-handoff type is 400.
This is graceful publish, not task kill, and it does not terminate
supervisors. Only groups that have a running task on that worker are queued.
Existing supervisor `taskGroups/handoff`, disable/enable, and
`shutdownAllTasks` are unchanged.
```mermaid
flowchart LR
client[Client] --> disable["POST worker/host/disable"]
client --> handoff["POST worker/host/handoff NEW"]
handoff --> groups["group supervisorId plus taskGroupId"]
groups --> notice["existing handoffTaskGroupsEarly"]
```
#### Release note
Added `POST /druid/indexer/v1/worker/{host}/handoff` to queue early handoff
of streaming tasks (`index_kafka`, `index_kinesis`) running on one Middle
Manager, using the same worker host as enable/disable.
##### Key changed/added classes in this PR
* `OverlordResource`
* `WorkerHandoffHelper`
* `WorkerHandoffRequest`
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
- [x] a release note entry in the PR description.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths.
## What changes
Before, draining a Middle Manager required listing tasks, fetching payloads,
and posting per-supervisor group ids. After, one Overlord call on that worker
host queues early handoff for its streaming tasks.
## Summary
Worker-scoped early handoff so operators do not reconstruct supervisor group
lists, and so HA supervisors whose id differs from the datasource still get the
correct notice.
## Test plan
- Unit tests in `WorkerHandoffHelperTest` and
`OverlordResourceWorkerHandoffTest`: two supervisors on one table, empty worker
202, unknown host 404, types with no handoff-capable value 400, skip compact /
empty supervisorId.
- Existing `OverlordResourceTest` enable/disable/shutdown tests unchanged;
added a case for a non-`WorkerTaskRunner`.
- Focused `indexing-service` tests should be run on CI (local full reactor
was not run on this laptop).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
