It looks like all of our tests using default ports are failing in the nightly build. Here is my suspicion of what happened: We have some Acceptance Tests that spin up separate JVMs for Locators and Servers. Those tests make sure to kill their child processes as part of their cleanup, but I suspect that when the build times out our test process gets hard-killed, which would prevent the proper cleanup from taking place. Does anyone have ssh access into this box to look for orphaned Java processes?
Thanks, Jared > Begin forwarded message: > > From: Apache Jenkins Server <jenk...@builds.apache.org> > Subject: Build failed in Jenkins: Geode-nightly #923 > Date: August 14, 2017 at 8:53:03 PM PDT > To: dev@geode.apache.org, kmil...@pivotal.io, kh...@apache.org, > bschucha...@pivotal.io, jstew...@pivotal.io, kl...@pivotal.io, > dschnei...@pivotal.io, dbar...@pivotal.io, n...@pivotal.io > Reply-To: dev@geode.apache.org > > See > <https://builds.apache.org/job/Geode-nightly/923/display/redirect?page=changes> > > Changes: > > [jiliao] GEODE-3328: fix a test failure on windows. > > [klund] Revert "GEODE-3328: fix a test failure on windows." > > [klund] GEODE-3436: revert recent refactoring of GFSH commands > > ------------------------------------------ > [...truncated 129.88 KB...] > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call > in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > Caused by: > java.net.BindException: Failed to create server socket on null[40,404] > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712) > at > org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > at > org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > Caused by: > java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779) > ... 5 more > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testSanity FAILED > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call > in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > Caused by: > java.net.BindException: Failed to create server socket on null[40,404] > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712) > at > org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > at > org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > Caused by: > java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779) > ... 5 more > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testInvalidate FAILED > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call > in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > Caused by: > java.net.BindException: Failed to create server socket on null[40,404] > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712) > at > org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > at > org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > Caused by: > java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779) > ... 5 more > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testRemoveAttribute FAILED > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call > in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > Caused by: > java.net.BindException: Failed to create server socket on null[40,404] > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712) > at > org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > at > org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > Caused by: > java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779) > ... 5 more > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testSessionExpirationByContainer FAILED > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call > in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > Caused by: > java.net.BindException: Failed to create server socket on null[40,404] > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712) > at > org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > at > org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > Caused by: > java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779) > ... 5 more > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testSessionPersists1 FAILED > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call > in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > Caused by: > java.net.BindException: Failed to create server socket on null[40,404] > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712) > at > org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > at > org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344) > at > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > Caused by: > java.net.BindException: Address already in use (Bind failed) > at java.net.PlainSocketImpl.socketBind(Native Method) > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) > at java.net.ServerSocket.bind(ServerSocket.java:375) > at > org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779) > ... 5 more > > 30 tests completed, 15 failed > :extensions/geode-modules-tomcat8:distributedTest FAILED > :extensions/geode-modules-tomcat8:integrationTest > :extensions/session-testing-war:javadoc > :extensions/session-testing-war:javadocJar > :extensions/session-testing-war:sourcesJar > :extensions/session-testing-war:war > :extensions/session-testing-war:signArchives SKIPPED > :extensions/session-testing-war:assemble > :extensions/session-testing-war:compileTestJava UP-TO-DATE > :extensions/session-testing-war:processTestResources UP-TO-DATE > :extensions/session-testing-war:testClasses UP-TO-DATE > :extensions/session-testing-war:checkMissedTests UP-TO-DATE > :extensions/session-testing-war:spotlessJavaCheck > :extensions/session-testing-war:spotlessCheck > :extensions/session-testing-war:test UP-TO-DATE > :extensions/session-testing-war:check > :extensions/session-testing-war:build > :extensions/session-testing-war:distributedTest UP-TO-DATE > :extensions/session-testing-war:integrationTest UP-TO-DATE > :geode-assembly:distTar > :geode-assembly:distZip > :geode-assembly:writeBuildInfo > :geode-assembly:srcDistTar > :geode-assembly:srcDistZip > :geode-assembly:signArchives SKIPPED > :geode-assembly:assemble > :geode-assembly:checkMissedTests > :geode-assembly:spotlessJavaCheck > :geode-assembly:spotlessCheck > :geode-assembly:test > :geode-assembly:check > :geode-assembly:build > :geode-assembly:distributedTest > > org.apache.geode.rest.internal.web.controllers.RestAPIsWithSSLDUnitTest > > testSSLWithCipherSuite[1] FAILED > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.test.dunit.NamedRunnable.run in VM 3 running on Host > asf905.gq1.ygridcore.net with 4 VMs > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > at org.apache.geode.test.dunit.VM.invoke(VM.java:290) > at > org.apache.geode.rest.internal.web.controllers.RestAPIsWithSSLDUnitTest.startInfraWithSSL(RestAPIsWithSSLDUnitTest.java:257) > at > org.apache.geode.rest.internal.web.controllers.RestAPIsWithSSLDUnitTest.testSSLWithCipherSuite(RestAPIsWithSSLDUnitTest.java:651) > > Caused by: > org.apache.geode.ToDataException: toData failed on PdxSerializable > class org.apache.geode.rest.internal.web.controllers.Person > at > org.apache.geode.internal.InternalDataSerializer.writePdx(InternalDataSerializer.java:3007) > at > org.apache.geode.internal.InternalDataSerializer.basicWriteObject(InternalDataSerializer.java:2077) > at > org.apache.geode.DataSerializer.writeObject(DataSerializer.java:2936) > at > org.apache.geode.internal.util.BlobHelper.serializeTo(BlobHelper.java:66) > at > org.apache.geode.internal.cache.tier.sockets.Message.serializeAndAddPart(Message.java:396) > at > org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:340) > at > org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:319) > at > org.apache.geode.cache.client.internal.PutOp$PutOpImpl.<init>(PutOp.java:281) > at > org.apache.geode.cache.client.internal.PutOp.execute(PutOp.java:66) > at > org.apache.geode.cache.client.internal.ServerRegionProxy.put(ServerRegionProxy.java:159) > at > org.apache.geode.internal.cache.LocalRegion.serverPut(LocalRegion.java:2987) > at > org.apache.geode.internal.cache.LocalRegion.cacheWriteBeforePut(LocalRegion.java:3096) > at > org.apache.geode.internal.cache.ProxyRegionMap.basicPut(ProxyRegionMap.java:222) > at > org.apache.geode.internal.cache.LocalRegion.virtualPut(LocalRegion.java:5598) > at > org.apache.geode.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:151) > at > org.apache.geode.internal.cache.LocalRegion.basicPut(LocalRegion.java:5027) > at > org.apache.geode.internal.cache.LocalRegion.validatedPut(LocalRegion.java:1580) > at > org.apache.geode.internal.cache.LocalRegion.put(LocalRegion.java:1567) > at > org.apache.geode.internal.cache.AbstractRegion.put(AbstractRegion.java:325) > at > org.apache.geode.rest.internal.web.controllers.RestAPIsWithSSLDUnitTest.doPutsInClientCache(RestAPIsWithSSLDUnitTest.java:184) > at > org.apache.geode.rest.internal.web.controllers.RestAPIsWithSSLDUnitTest.lambda$startInfraWithSSL$1253249a$4(RestAPIsWithSSLDUnitTest.java:257) > > Caused by: > org.apache.geode.cache.client.ServerConnectivityException: Pool > unexpected socket timed out on client connection=Pooled Connection to > asf905.gq1.ygridcore.net:52966: > Connection[asf905.gq1.ygridcore.net:52966]@1232456419). Server unreachable: > could not connect after 1 attempts > at > org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:798) > at > org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:623) > at > org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:174) > at > org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:115) > at > org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:775) > at > org.apache.geode.cache.client.internal.GetPDXIdForEnumOp.execute(GetPDXIdForEnumOp.java:38) > at > org.apache.geode.pdx.internal.ClientTypeRegistration.processEnumInfoForEnumId(ClientTypeRegistration.java:161) > at > org.apache.geode.pdx.internal.ClientTypeRegistration.getEnumId(ClientTypeRegistration.java:153) > at > org.apache.geode.pdx.internal.TypeRegistry.getEnumId(TypeRegistry.java:367) > at > org.apache.geode.internal.InternalDataSerializer.writePdxEnum(InternalDataSerializer.java:2139) > at > org.apache.geode.internal.InternalDataSerializer.writeUserObject(InternalDataSerializer.java:1529) > at > org.apache.geode.internal.InternalDataSerializer.writeWellKnownObject(InternalDataSerializer.java:1437) > at > org.apache.geode.internal.InternalDataSerializer.basicWriteObject(InternalDataSerializer.java:2102) > at > org.apache.geode.pdx.internal.PdxOutputStream.writeObject(PdxOutputStream.java:82) > at > org.apache.geode.pdx.internal.PdxWriterImpl.writeObject(PdxWriterImpl.java:329) > at > org.apache.geode.pdx.internal.PdxWriterImpl.writeObject(PdxWriterImpl.java:319) > at > org.apache.geode.rest.internal.web.controllers.Person.toData(Person.java:166) > at > org.apache.geode.internal.InternalDataSerializer.writePdx(InternalDataSerializer.java:2977) > ... 20 more > Build timed out (after 1,440 minutes). Marking the build as failed. > Build was aborted > Archiving artifacts > [Fast Archiver] Compressed 338.72 MB of artifacts by 42.4% relative to #914 > Recording test results > Not sending mail to unregistered user ukohlme...@pivotal.io > Not sending mail to unregistered user gz...@pivotal.io > Not sending mail to unregistered user lhughesgodf...@pivotal.io > Not sending mail to unregistered user e...@pivotal.io > Not sending mail to unregistered user huyn...@gmail.com > Not sending mail to unregistered user hkhame...@pivotal.io > Not sending mail to unregistered user jil...@pivotal.io
