yashmayya commented on code in PR #15005:
URL: https://github.com/apache/pinot/pull/15005#discussion_r1954115183
##########
pinot-common/src/main/java/org/apache/pinot/common/failuredetector/BaseExponentialBackoffRetryFailureDetector.java:
##########
@@ -88,21 +102,28 @@ public void start() {
LOGGER.info("Server: {} has been marked healthy, skipping the
retry", instanceId);
continue;
}
- if (retryInfo._numRetires == _maxRetries) {
+ if (retryInfo._numRetries == _maxRetries) {
LOGGER.warn("Unhealthy server: {} already reaches the max retries:
{}, do not retry again and treat it "
+ "as healthy so that the listeners do not lose track of the
server", instanceId, _maxRetries);
markServerHealthy(instanceId);
continue;
}
LOGGER.info("Retry unhealthy server: {}", instanceId);
- for (Listener listener : _listeners) {
- listener.retryUnhealthyServer(instanceId, this);
+ boolean recovered = true;
+ for (Function<String, Boolean> unhealthyServerRetrier :
_unhealthyServerRetriers) {
+ if (!unhealthyServerRetrier.apply(instanceId)) {
+ recovered = false;
Review Comment:
Yeah with the previous implementation, it was possible for multiple
listeners to mark the same server as healthy on a call to
`retryUnhealthyServer`. I've updated the implementation so that the failure
detector itself marks the server as healthy if all the listeners / retriers
report that the server is healthy again (to be safe).
##########
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/GrpcBrokerRequestHandler.java:
##########
@@ -147,4 +162,26 @@ public void shutdown() {
}
}
}
+
+ /**
+ * Check if a server that was previously detected as unhealthy is now
healthy.
+ */
+ public boolean retryUnhealthyServer(String instanceId) {
Review Comment:
Oops, my bad, left over from a previous iteration with a different rewrite
implementation. I've update it to `private`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
