[
https://issues.apache.org/jira/browse/GEODE-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirk Lund updated GEODE-1165:
-----------------------------
Comment: was deleted
(was: A Pivotal Tracker story has been created for this Issue:
https://www.pivotaltracker.com/story/show/138521697)
> CI failure: SharedConfigurationUsingDirDUnitTest tests are flaky
> ----------------------------------------------------------------
>
> Key: GEODE-1165
> URL: https://issues.apache.org/jira/browse/GEODE-1165
> Project: Geode
> Issue Type: Bug
> Components: configuration
> Affects Versions: 1.0.0-incubating
> Reporter: Jianxia Chen
> Assignee: Kirk Lund
> Labels: BindException, ClusterConfig, Flaky
>
> Several tests in SharedConfigurationUsingDirDUnitTest are flaky and fail
> intermittently. One known cause is GEODE-2238 (one cause has been fixed and
> committed). The timeout used by these tests is 15 seconds which is quite low
> if a GC pause occurs.
> 1) GEODE-1165 includes BindException:
> {noformat}
> com.gemstone.gemfire.test.dunit.RMIException: While invoking
> com.gemstone.gemfire.test.dunit.NamedRunnable.run in VM 1 running on Host
> cc2-ub.gemstone.com with 4 VMs
> at com.gemstone.gemfire.test.dunit.VM.invoke(VM.java:439)
> at com.gemstone.gemfire.test.dunit.VM.invoke(VM.java:381)
> at com.gemstone.gemfire.test.dunit.VM.invoke(VM.java:303)
> at
> com.gemstone.gemfire.management.internal.configuration.SharedConfigurationUsingDirDUnitTest.startLocator(SharedConfigurationUsingDirDUnitTest.java:266)
> at
> com.gemstone.gemfire.management.internal.configuration.SharedConfigurationUsingDirDUnitTest.updateClusterConfigDirWithTwoLocatorsNoRollingServerRestart(SharedConfigurationUsingDirDUnitTest.java:122)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:105)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:56)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at
> org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
> at
> org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at
> org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
> at
> org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Failed to create server socket on
> null[23,762]
> at
> com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:846)
> at
> com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:806)
> at
> com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:768)
> at
> com.gemstone.gemfire.distributed.internal.tcpserver.TcpServer.startServerThread(TcpServer.java:193)
> at
> com.gemstone.gemfire.distributed.internal.tcpserver.TcpServer.start(TcpServer.java:187)
> at
> com.gemstone.gemfire.distributed.internal.InternalLocator.startTcpServer(InternalLocator.java:620)
> at
> com.gemstone.gemfire.distributed.internal.InternalLocator.startPeerLocation(InternalLocator.java:673)
> at
> com.gemstone.gemfire.distributed.internal.InternalLocator.startLocator(InternalLocator.java:395)
> at
> com.gemstone.gemfire.distributed.internal.InternalLocator.startLocator(InternalLocator.java:328)
> at
> com.gemstone.gemfire.distributed.Locator.startLocator(Locator.java:339)
> at
> com.gemstone.gemfire.distributed.Locator.startLocatorAndDS(Locator.java:171)
> at
> com.gemstone.gemfire.management.internal.configuration.SharedConfigurationUsingDirDUnitTest.lambda$startLocator$e22b6e8a$1(SharedConfigurationUsingDirDUnitTest.java:280)
> at
> com.gemstone.gemfire.test.dunit.NamedRunnable.run(NamedRunnable.java:33)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at hydra.MethExecutor.executeObject(MethExecutor.java:268)
> at
> com.gemstone.gemfire.test.dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:83)
> at sun.reflect.GeneratedMethodAccessor139.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$95(TCPTransport.java:683)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> ... 3 more
> Caused by: java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
> at java.net.ServerSocket.bind(ServerSocket.java:375)
> at
> com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:843)
> ... 34 more
> {noformat}
> 2) GEODE-1979 included ConditionTimeoutException:
> {noformat}
> org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.test.dunit.NamedRunnable.run in VM 3 running on Host
> cc1-co.gemstone.com with 4 VMs
> at org.apache.geode.test.dunit.VM.invoke(VM.java:389)
> at org.apache.geode.test.dunit.VM.invoke(VM.java:355)
> at org.apache.geode.test.dunit.VM.invoke(VM.java:280)
> at
> org.apache.geode.management.internal.configuration.SharedConfigurationUsingDirDUnitTest.basicClusterConfigDirWithTwoLocators(SharedConfigurationUsingDirDUnitTest.java:108)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:114)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:57)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at sun.reflect.GeneratedMethodAccessor300.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109)
> at sun.reflect.GeneratedMethodAccessor299.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:377)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
> at
> org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.jayway.awaitility.core.ConditionTimeoutException: Condition
> with lambda expression in
> org.apache.geode.management.internal.configuration.SharedConfigurationUsingDirDUnitTest
> was not fulfilled within 15 seconds.
> at
> com.jayway.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:122)
> at
> com.jayway.awaitility.core.CallableCondition.await(CallableCondition.java:79)
> at
> com.jayway.awaitility.core.CallableCondition.await(CallableCondition.java:27)
> at
> com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:764)
> at
> com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:741)
> at
> org.apache.geode.management.internal.configuration.SharedConfigurationUsingDirDUnitTest.lambda$basicClusterConfigDirWithTwoLocators$bb17a952$1(SharedConfigurationUsingDirDUnitTest.java:109)
> at org.apache.geode.test.dunit.NamedRunnable.run(NamedRunnable.java:33)
> at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at hydra.MethExecutor.executeObject(MethExecutor.java:268)
> at
> org.apache.geode.test.dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:82)
> at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$95(TCPTransport.java:683)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> ... 3 more
> Previously run tests: [ConfigCommandsDUnitTest, DiskStoreCommandsDUnitTest,
> GemfireDataCommandsDUnitTest, IndexCommandsDUnitTest,
> MiscellaneousCommandsDUnitTest,
> MiscellaneousCommandsExportLogsPart1DUnitTest,
> MiscellaneousCommandsExportLogsPart4DUnitTest, QueueCommandsDUnitTest,
> ShowMetricsDUnitTest, SharedConfigurationDUnitTest,
> SharedConfigurationUsingDirDUnitTest]
> {noformat}
> 3) GEODE-1989 included ConditionTimeoutException as well:
> {noformat}
> org.apache.geode.management.internal.configuration.SharedConfigurationUsingDirDUnitTest
> > updateClusterConfigDirWithTwoLocatorsAndRollingServerRestart FAILED
> org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.test.dunit.NamedRunnable.run in VM 2 running on Host
> a9e7f8bc8ba0 with 4 VMs
> at org.apache.geode.test.dunit.VM.invoke(VM.java:389)
> at org.apache.geode.test.dunit.VM.invoke(VM.java:355)
> at org.apache.geode.test.dunit.VM.invoke(VM.java:280)
> at
> org.apache.geode.management.internal.configuration.SharedConfigurationUsingDirDUnitTest.updateClusterConfigDirWithTwoLocatorsAndRollingServerRestart(SharedConfigurationUsingDirDUnitTest.java:208)
> Caused by:
> com.jayway.awaitility.core.ConditionTimeoutException: Condition with lambda
> expression in
> org.apache.geode.management.internal.configuration.SharedConfigurationUsingDirDUnitTest
> was not fulfilled within 15 seconds.
> at
> com.jayway.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:122)
> at
> com.jayway.awaitility.core.CallableCondition.await(CallableCondition.java:79)
> at
> com.jayway.awaitility.core.CallableCondition.await(CallableCondition.java:27)
> at
> com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:764)
> at
> com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:741)
> at
> org.apache.geode.management.internal.configuration.SharedConfigurationUsingDirDUnitTest.lambda$updateClusterConfigDirWithTwoLocatorsAndRollingServerRestart$bb17a952$2(SharedConfigurationUsingDirDUnitTest.java:209)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
