somandal commented on code in PR #15266:
URL: https://github.com/apache/pinot/pull/15266#discussion_r2023472442
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/ZkBasedTableRebalanceObserver.java:
##########
@@ -68,31 +70,67 @@ public ZkBasedTableRebalanceObserver(String
tableNameWithType, String rebalanceJ
@Override
public void onTrigger(Trigger trigger, Map<String, Map<String, String>>
currentState,
- Map<String, Map<String, String>> targetState) {
+ Map<String, Map<String, String>> targetState, RebalanceContext
rebalanceContext) {
boolean updatedStatsInZk = false;
_controllerMetrics.setValueOfTableGauge(_tableNameWithType,
ControllerGauge.TABLE_REBALANCE_IN_PROGRESS, 1);
+ TableRebalanceProgressStats.RebalanceStateStats latest;
+ TableRebalanceProgressStats.RebalanceProgressStats latestProgress;
switch (trigger) {
case START_TRIGGER:
- updateOnStart(currentState, targetState);
+ updateOnStart(currentState, targetState, rebalanceContext);
trackStatsInZk();
updatedStatsInZk = true;
break;
// Write to Zk if there's change since previous stats computation
case IDEAL_STATE_CHANGE_TRIGGER:
- TableRebalanceProgressStats.RebalanceStateStats latest =
- getDifferenceBetweenTableRebalanceStates(targetState,
currentState);
+ latest = getDifferenceBetweenTableRebalanceStates(targetState,
currentState);
+ latestProgress = calculateOverallProgressStats(targetState,
+ currentState, rebalanceContext,
Trigger.IDEAL_STATE_CHANGE_TRIGGER, _tableRebalanceProgressStats);
if
(TableRebalanceProgressStats.statsDiffer(_tableRebalanceProgressStats.getCurrentToTargetConvergence(),
- latest)) {
- _tableRebalanceProgressStats.setCurrentToTargetConvergence(latest);
+ latest) || TableRebalanceProgressStats.progressStatsDiffer(
+ _tableRebalanceProgressStats.getRebalanceProgressStatsOverall(),
latestProgress)) {
+ if (TableRebalanceProgressStats.statsDiffer(
+
_tableRebalanceProgressStats.getExternalViewToIdealStateConvergence(), latest))
{
+ _tableRebalanceProgressStats.setCurrentToTargetConvergence(latest);
Review Comment:
it should technically not do that. earlier it was possible for scenarios
where `bestEffort=true`, but I added handling for that kind of overflow. I am
also trying to track newly added segments that aren't tracked by rebalance
separately etc. If we do see weird behavior it means there is some bug that
needs to be fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
