[
https://issues.apache.org/jira/browse/GEODE-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015269#comment-17015269
]
Dan Smith commented on GEODE-5816:
----------------------------------
I looked at this latest run from Darrel. It looks like it is still picking the
same port for the jmx and http services.
The PortSupplier logic that tries to prevent duplicate ports fails due to the
`locatorProperties` method I mentioned earlier, which bypasses the port
supplier. However, I'm not sure why Brian's other fix to AvailablePortHelper
itself didn't fix this problem.
> ClusterStartupRule fails to launch JMX manager (port already in use)
> --------------------------------------------------------------------
>
> Key: GEODE-5816
> URL: https://issues.apache.org/jira/browse/GEODE-5816
> Project: Geode
> Issue Type: Bug
> Components: tests
> Reporter: Bill Burcham
> Assignee: Dan Smith
> Priority: Major
> Labels: CI, swat
>
> We see these related failures on a couple tests in
> RegionMembershipMBeanOverHttpDUnitTest from our recent mass-test-run.
> From looking at the stack traces though, we surmise that the problem occurs
> in the ClusterStartupRule _before_ the actual tests run. Since it occurs
> before the tests run, we think the problem lies outside the
> RegionMembershipMBeanOverHttpDUnitTest class.
> {noformat}
> 4 failures (99.600% success
> rate)org.apache.geode.management.internal.cli.commands.RegionMembershipMBeanOverHttpDUnitTest
> | .testAddRmNewMemberWithReplicatedRegionsAndSubregions: 1 failures
> (99.900% success rate)
> | | Failed build 24 at
> https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/24
> | .testMultiplePartitionedRegions: 3 failures (99.700% success rate)
> | | Failed build 982 at
> https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/982
> | | Failed build 256 at
> https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/256
> {noformat}
> Here's a stack trace:
> {noformat}
> java.lang.AssertionError: Suspicious strings were written to the log during
> this run.
> Fix the strings or use IgnoredException.addIgnoredException to ignore.
> -----------------------------------------------------------------------
> Found suspect string in log4j at line 358
> [error 2018/10/01 06:28:30.869 UTC <RMI TCP Connection(1)-172.17.0.20>
> tid=0x20] Jmx manager could not be started because
> java.rmi.server.ExportException: Port already in use: 25305; nested exception
> is:
> java.net.BindException: Failed to create server socket on
> ffd50f3577c5/172.17.0.20[25305]
> org.apache.geode.management.ManagementException:
> java.rmi.server.ExportException: Port already in use: 25305; nested exception
> is:
> java.net.BindException: Failed to create server socket on
> ffd50f3577c5/172.17.0.20[25305]
> at
> org.apache.geode.management.internal.ManagementAgent.startAgent(ManagementAgent.java:162)
> at
> org.apache.geode.management.internal.SystemManagementService.startManager(SystemManagementService.java:435)
> at
> org.apache.geode.management.internal.beans.ManagementAdapter.handleCacheCreation(ManagementAdapter.java:173)
> at
> org.apache.geode.management.internal.beans.ManagementListener.handleEvent(ManagementListener.java:118)
> at
> org.apache.geode.distributed.internal.InternalDistributedSystem.notifyResourceEventListeners(InternalDistributedSystem.java:2201)
> at
> org.apache.geode.distributed.internal.InternalDistributedSystem.handleResourceEvent(InternalDistributedSystem.java:591)
> at
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1218)
> at
> org.apache.geode.internal.cache.GemFireCacheImpl.basicCreate(GemFireCacheImpl.java:793)
> at
> org.apache.geode.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:779)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:177)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:224)
> at
> org.apache.geode.distributed.internal.InternalLocator.startCache(InternalLocator.java:662)
> at
> org.apache.geode.distributed.internal.InternalLocator.startDistributedSystem(InternalLocator.java:649)
> at
> org.apache.geode.distributed.internal.InternalLocator.startLocator(InternalLocator.java:311)
> at org.apache.geode.distributed.Locator.startLocator(Locator.java:253)
> at
> org.apache.geode.distributed.Locator.startLocatorAndDS(Locator.java:140)
> at
> org.apache.geode.test.junit.rules.LocatorStarterRule.startLocator(LocatorStarterRule.java:87)
> at
> org.apache.geode.test.junit.rules.LocatorStarterRule.before(LocatorStarterRule.java:68)
> at
> org.apache.geode.test.dunit.rules.ClusterStartupRule.lambda$startLocatorVM$22d9b8a8$1(ClusterStartupRule.java:206)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at hydra.MethExecutor.executeObject(MethExecutor.java:244)
> at
> org.apache.geode.test.dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:70)
> at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.rmi.server.ExportException: Port already in use: 25305;
> nested exception is:
> java.net.BindException: Failed to create server socket on
> ffd50f3577c5/172.17.0.20[25305]
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:346)
> at
> sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:254)
> at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> at
> sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:236)
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213)
> at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:173)
> at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:144)
> at
> java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
> at
> org.apache.geode.management.internal.ManagementAgent.configureAndStart(ManagementAgent.java:456)
> at
> org.apache.geode.management.internal.ManagementAgent.startAgent(ManagementAgent.java:160)
> ... 40 more
> Caused by: java.net.BindException: Failed to create server socket on
> ffd50f3577c5/172.17.0.20[25305]
> at
> org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:758)
> at
> org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:715)
> at
> org.apache.geode.management.internal.ManagementAgent$GemFireRMIServerSocketFactory.createServerSocket(ManagementAgent.java:615)
> at
> sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335)
> ... 50 more
> Caused by: java.net.BindException: Address already in use (Bind failed)
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
> at java.net.ServerSocket.bind(ServerSocket.java:375)
> at
> org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:754)
> ... 54 more
> -----------------------------------------------------------------------
> Found suspect string in log4j at line 431
> [error 2018/10/01 06:28:30.872 UTC <RMI TCP Connection(1)-172.17.0.20>
> tid=0x20] org.apache.geode.management.ManagementException:
> java.rmi.server.ExportException: Port already in use: 25305; nested exception
> is:
> java.net.BindException: Failed to create server socket on
> ffd50f3577c5/172.17.0.20[25305]
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.geode.test.dunit.standalone.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:411)
> at
> org.apache.geode.test.dunit.rules.ClusterStartupRule.after(ClusterStartupRule.java:172)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
> at
> org.apache.geode.test.junit.rules.DescribedExternalResource$1.evaluate(DescribedExternalResource.java:40)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:137)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
> at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The three stack traces are almost identical.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
