gortiz commented on code in PR #12345:
URL: https://github.com/apache/pinot/pull/12345#discussion_r1474102380
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/periodictask/ControllerPeriodicTask.java:
##########
@@ -63,29 +69,25 @@ protected final void runTask(Properties
periodicTaskProperties) {
// Check if we have a specific table against which this task needs to be
run.
String propTableNameWithType = (String)
periodicTaskProperties.get(PeriodicTask.PROPERTY_KEY_TABLE_NAME);
// Process the tables that are managed by this controller
- List<String> tablesToProcess = new ArrayList<>();
- List<String> nonLeaderForTables = new ArrayList<>();
- if (propTableNameWithType == null) {
- // Table name is not available, so task should run on all tables for
which this controller is the lead.
- for (String tableNameWithType :
_pinotHelixResourceManager.getAllTables()) {
- if (_leadControllerManager.isLeaderForTable(tableNameWithType)) {
- tablesToProcess.add(tableNameWithType);
- } else {
- nonLeaderForTables.add(tableNameWithType);
- }
- }
- } else {
- // Table name is available, so task should run only on the specified
table.
- if (_leadControllerManager.isLeaderForTable(propTableNameWithType)) {
- tablesToProcess.add(propTableNameWithType);
- }
- }
+ Set<String> allTables = propTableNameWithType == null
+ ? new HashSet<>(_pinotHelixResourceManager.getAllTables())
+ : Collections.singleton(propTableNameWithType);
+
+ Set<String> currentLeaderOfTables = allTables.stream()
+ .filter(_leadControllerManager::isLeaderForTable)
+ .collect(Collectors.toSet());
- if (!tablesToProcess.isEmpty()) {
- processTables(tablesToProcess, periodicTaskProperties);
+ if (!currentLeaderOfTables.isEmpty()) {
+ processTables(new ArrayList<>(currentLeaderOfTables),
periodicTaskProperties);
}
+
+ Sets.SetView<String> allKnownTables = Sets.union(_prevLeaderOfTables,
allTables);
+ Set<String> nonLeaderForTables = Sets.difference(allKnownTables,
currentLeaderOfTables);
Review Comment:
Javadoc of `nonLeaderCleanup` says: `Can be overridden to perform cleanups
for tables that the current controller isn't the leader.`
Where the key part is `tables that the current controller isn't the leader`.
That includes tables the current controller is not leader and wasn't the leader
in the previous iteration. The expression `_prevLeaderOfTables -
currentLeaderOfTables` does only return the tables we ended up not being leader
since previous iteration.
I guess it is a subtle difference and probably the current code only cares
about the tables we are no longer leader of. I can apply that change, but I
wanted to be formal on the semantics in case there are third party extensions
that are actually based on the javadoc contract.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
