mjsax commented on code in PR #16922:
URL: https://github.com/apache/kafka/pull/16922#discussion_r1735449052
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java:
##########
@@ -454,6 +479,24 @@ private void handleTasksWithoutStateUpdater(final
Map<TaskId, Set<TopicPartition
final Map<TaskId,
Set<TopicPartition>> standbyTasksToCreate,
final Map<Task,
Set<TopicPartition>> tasksToRecycle,
final Set<Task>
tasksToCloseClean) {
+ final Map<Task, Set<TopicPartition>> initialStandbyTasksToRecycle =
assignInitialTasks(activeTasksToCreate, logPrefix, topologyMetadata,
changelogReader);
+ final Map<Task, Set<TopicPartition>> initialStandbyTasksToUse =
assignInitialTasks(activeTasksToCreate, logPrefix, topologyMetadata,
changelogReader);
+ final Set<TaskId> recycledInitialTasks = new
HashSet<>(initialStandbyTasksToRecycle.size() +
initialStandbyTasksToUse.size());
+
+ // if this was the last local thread to receive its assignment, close
all the remaining Tasks, as they are not needed
+ stateDirectory.closeInitialTasksIfLastAssginedThread();
+
+ // recycle the initial standbys to active, and remove them from the
set of actives that need to be created
+ tasksToRecycle.putAll(initialStandbyTasksToRecycle);
Review Comment:
> We have to do some extra book-keeping to ensure we don't create new Tasks
that we already have "pending" standbys for.
Guess this is the part I don't see? Why do we need to expliclity add them to
`tasksToRecycle` and remove from `activeTasksToCreate`? After we add them via
`tasks.addStandbyTasks(initialStandbyTasksToRecycle.keySet());` the for-loop
below should do everything else we need? Or what do I miss?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
