[ https://issues.apache.org/jira/browse/GEODE-9025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakov Varenina updated GEODE-9025: ---------------------------------- Description: When running Apache geode in Kubernetes, then in some cases ClassCastException is thrown when locator discovery is performed. This exception occurs when locator try to cast received Object to RemoteLocatorJoinResponse object. The problem is that locator discovery thread is then stopped, and due to that, locator discovery will never be successfully performed. The only way to trigger locator discovery again is to restart the locator. *The root cause of this issues is following:* If locator gets EOFException when sending VersionRequest message, then it automatically assumes that remote locator is running old version of geode which doesn't support VersionRequest. Locator then uses the oldest known version and sends RemoteLocatorJoinRequest towards the remote locator. Then locator tries to read the response as follows: {code:java} public Object requestToServer(HostAndPort addr, Object request, int timeout, boolean replyExpected) throws IOException, ClassNotFoundException { ... Object response = objectDeserializer.readObject(versionedDataInputStream); logger.debug("received response: {}", response); return response; } {code} Because locator reads response as Object, that means, it will read any type of object. The function requestToServer() will then return object which will cause ClassCastException, since it is not of type LocatorRequestJoinResponse: {code:java} public void exchangeRemoteLocators() { ... RemoteLocatorJoinResponse response = (RemoteLocatorJoinResponse) locatorClient .requestToServer(locatorId.getHost(), request, WAN_LOCATOR_CONNECTION_TIMEOUT, true); {code} *The solution:* > ClassCastException occures during remote locator discovery > ---------------------------------------------------------- > > Key: GEODE-9025 > URL: https://issues.apache.org/jira/browse/GEODE-9025 > Project: Geode > Issue Type: Bug > Reporter: Jakov Varenina > Assignee: Jakov Varenina > Priority: Major > > When running Apache geode in Kubernetes, then in some cases > ClassCastException is thrown when locator discovery is performed. This > exception occurs when locator try to cast received Object to > RemoteLocatorJoinResponse object. The problem is that locator discovery > thread is then stopped, and due to that, locator discovery will never be > successfully performed. The only way to trigger locator discovery again is to > restart the locator. > *The root cause of this issues is following:* > If locator gets EOFException when sending VersionRequest message, then it > automatically assumes that remote locator is running old version of geode > which doesn't support VersionRequest. Locator then uses the oldest known > version and sends RemoteLocatorJoinRequest towards the remote locator. Then > locator tries to read the response as follows: > {code:java} > public Object requestToServer(HostAndPort addr, Object request, int timeout, > boolean replyExpected) throws IOException, ClassNotFoundException { > ... > Object response = > objectDeserializer.readObject(versionedDataInputStream); > logger.debug("received response: {}", response); > return response; > } > {code} > Because locator reads response as Object, that means, it will read any type > of object. The function requestToServer() will then return object which will > cause ClassCastException, since it is not of type LocatorRequestJoinResponse: > {code:java} > public void exchangeRemoteLocators() { > ... > RemoteLocatorJoinResponse response = (RemoteLocatorJoinResponse) > locatorClient > .requestToServer(locatorId.getHost(), request, > WAN_LOCATOR_CONNECTION_TIMEOUT, true); > {code} > > *The solution:* > > -- This message was sent by Atlassian Jira (v8.3.4#803005)