J-HowHuang commented on issue #16955: URL: https://github.com/apache/pinot/issues/16955#issuecomment-3367201821
``` 2025-10-03T19:00:13.0859824Z 19:00:13.082 ERROR [ZkBasedTenantRebalanceObserver] [main] Error updating ZK for jobId: b5ed3948-19b8-4f5f-a0cb-af361e46051a on starting tenant rebalance 2025-10-03T19:00:13.0862281Z org.apache.pinot.spi.utils.retry.AttemptsExceededException: Operation failed after 3 attempts 2025-10-03T19:00:13.0863895Z at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:65) 2025-10-03T19:00:13.0867004Z at org.apache.pinot.controller.helix.core.rebalance.tenant.ZkBasedTenantRebalanceObserver.updateTenantRebalanceJobMetadataInZk(ZkBasedTenantRebalanceObserver.java:260) 2025-10-03T19:00:13.0869573Z at org.apache.pinot.controller.helix.core.rebalance.tenant.ZkBasedTenantRebalanceObserver.onStart(ZkBasedTenantRebalanceObserver.java:100) 2025-10-03T19:00:13.0871444Z at org.apache.pinot.controller.helix.core.rebalance.tenant.TenantRebalancer.rebalanceWithObserver(TenantRebalancer.java:195) 2025-10-03T19:00:13.0873098Z at org.apache.pinot.controller.helix.core.rebalance.tenant.TenantRebalancer.rebalance(TenantRebalancer.java:167) 2025-10-03T19:00:13.0874659Z at org.apache.pinot.controller.helix.core.rebalance.tenant.TenantRebalancerTest.testRebalanceBasic(TenantRebalancerTest.java:139) 2025-10-03T19:00:13.0876379Z at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) 2025-10-03T19:00:13.0877376Z at java.base/java.lang.reflect.Method.invoke(Method.java:580) ``` Looks like the updates failed due to the ZK updates attempts. I'm not sure the root cause of update failure here, it's less likely to be race condition on updating tenant rebalance metadata since there should be no other threads doing it. Could be the ZK property set failure per se. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
