[
https://issues.apache.org/jira/browse/GEODE-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550663#comment-17550663
]
Geode Integration commented on GEODE-10167:
-------------------------------------------
Seen in [upgrade-test-openjdk11
#386|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/upgrade-test-openjdk11/builds/386]
... see [test
results|http://files.apachegeode-ci.info/builds/apache-develop-main/1.16.0-build.0313/test-results/upgradeTest/1654510296/]
or download
[artifacts|http://files.apachegeode-ci.info/builds/apache-develop-main/1.16.0-build.0313/test-artifacts/1654510296/upgradetestfiles-openjdk11-1.16.0-build.0313.tgz].
> CI failure:
> RollingUpgradeQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled
> > luceneQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: GEODE-10167
> URL: https://issues.apache.org/jira/browse/GEODE-10167
> Project: Geode
> Issue Type: Bug
> Reporter: Jianxia Chen
> Priority: Major
>
> {code:java}
> RollingUpgradeQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled
> >
> luceneQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled[from_v1.8.0,
> with reindex=false, singleHopEnabled=true] FAILED
> 00:24:01 org.gradle.internal.exceptions.DefaultMultiCauseException:
> Multiple Failures (2 failures)
> 00:24:01 org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.cache.lucene.LuceneSearchWithRollingUpgradeTestBase$$Lambda$422/0x0000000100379c40.run
> in VM 2 running on Host
> heavy-lifter-138290eb-3fbe-5801-8565-60d3c24284e9.c.apachegeode-ci.internal
> with 4 VMs
> 00:24:01 java.lang.AssertionError: Suspicious strings were written to
> the log during this run.
> 00:24:01 Fix the strings or use IgnoredException.addIgnoredException to
> ignore.
> 00:24:01
> -----------------------------------------------------------------------
> 00:24:01 Found suspect string in 'dunit_suspect-vm2.log' at line 3243
> 00:24:01
> 00:24:01 [fatal 2022/03/25 07:23:55.137 UTC <unicast
> receiver,heavy-lifter-138290eb-3fbe-5801-8565-60d3c24284e9-23002> tid=49]
> Membership service failure: Member isn't responding to heartbeat requests
> 00:24:01
> org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException:
> Member isn't responding to heartbeat requests
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1806)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1120)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.processRemoveMemberMessage(GMSJoinLeave.java:723)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1367)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1303)
> 00:24:01 at org.jgroups.JChannel.invokeCallback(JChannel.java:816)
> 00:24:01 at org.jgroups.JChannel.up(JChannel.java:741)
> 00:24:01 at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
> 00:24:01 at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> 00:24:01 at org.jgroups.protocols.FlowControl.up(FlowControl.java:390)
> 00:24:01 at
> org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1077)
> 00:24:01 at
> org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:792)
> 00:24:01 at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.messenger.StatRecorder.up(StatRecorder.java:72)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.messenger.AddressManager.up(AddressManager.java:70)
> 00:24:01 at org.jgroups.protocols.TP.passMessageUp(TP.java:1658)
> 00:24:01 at
> org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1876)
> 00:24:01 at
> org.jgroups.util.DirectExecutor.execute(DirectExecutor.java:10)
> 00:24:01 at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1789)
> 00:24:01 at org.jgroups.protocols.TP.receive(TP.java:1714)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.messenger.Transport.receive(Transport.java:160)
> 00:24:01 at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:701)
> 00:24:01 at java.base/java.lang.Thread.run(Thread.java:829)
> 00:24:01 at
> org.junit.vintage.engine.execution.TestRun.getStoredResultOrSuccessful(TestRun.java:196)
> 00:24:01 at
> org.junit.vintage.engine.execution.RunListenerAdapter.fireExecutionFinished(RunListenerAdapter.java:226)
> 00:24:01 at
> org.junit.vintage.engine.execution.RunListenerAdapter.testFinished(RunListenerAdapter.java:192)
> 00:24:01 at
> org.junit.vintage.engine.execution.RunListenerAdapter.testFinished(RunListenerAdapter.java:79)
> 00:24:01 at
> org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87)
> 00:24:01 at
> org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225)
> 00:24:01 at
> org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
> 00:24:01 at
> org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222)
> 00:24:01 at
> org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38)
> 00:24:01 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372)
> 00:24:01 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> 00:24:01 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> 00:24:01 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> 00:24:01 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> 00:24:01 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> 00:24:01 at
> org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> 00:24:01 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> 00:24:01 at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> 00:24:01 at org.junit.runners.Suite.runChild(Suite.java:128)
> 00:24:01 at org.junit.runners.Suite.runChild(Suite.java:27)
> 00:24:01 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> 00:24:01 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> 00:24:01 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> 00:24:01 at
> org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> 00:24:01 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> 00:24:01 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 00:24:01 at
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> 00:24:01 at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> 00:24:01 at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> 00:24:01 at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
> 00:24:01 at
> org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
> 00:24:01 at
> org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
> 00:24:01 at
> org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
> 00:24:01 at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
> 00:24:01 at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
> 00:24:01 at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
> 00:24:01 at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
> 00:24:01 at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
> 00:24:01 at
> org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
> 00:24:01 at
> org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
> 00:24:01 at
> org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
> 00:24:01 at
> org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
> 00:24:01 at
> org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
> 00:24:01 at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
> 00:24:01 at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 00:24:01 at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 00:24:01 at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 00:24:01 at java.lang.reflect.Method.invoke(Method.java:566)
> 00:24:01 at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> 00:24:01 at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> 00:24:01 at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> 00:24:01 at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> 00:24:01 at com.sun.proxy.$Proxy2.stop(Unknown Source)
> 00:24:01 at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133)
> 00:24:01 at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 00:24:01 at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 00:24:01 at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 00:24:01 at java.lang.reflect.Method.invoke(Method.java:566)
> 00:24:01 at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> 00:24:01 at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> 00:24:01 at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> 00:24:01 at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> 00:24:01 at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
> 00:24:01 at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> 00:24:01 at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> 00:24:01 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> 00:24:01 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> 00:24:01 at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> 00:24:01 at java.lang.Thread.run(Thread.java:829)
> 00:24:01
> 00:24:01 Caused by:
> 00:24:01 org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.cache.lucene.LuceneSearchWithRollingUpgradeTestBase$$Lambda$422/0x0000000100379c40.run
> in VM 2 running on Host
> heavy-lifter-138290eb-3fbe-5801-8565-60d3c24284e9.c.apachegeode-ci.internal
> with 4 VMs
> 00:24:01 at
> org.apache.geode.test.dunit.VM.executeMethodOnObject(VM.java:631)
> 00:24:01 at org.apache.geode.test.dunit.VM.invoke(VM.java:448)
> 00:24:01 at
> org.apache.geode.cache.lucene.LuceneSearchWithRollingUpgradeTestBase.verifyLuceneQueryResultInEachVM(LuceneSearchWithRollingUpgradeTestBase.java:319)
> 00:24:01 at
> org.apache.geode.cache.lucene.LuceneSearchWithRollingUpgradeTestBase.putSerializableObjectAndVerifyLuceneQueryResult(LuceneSearchWithRollingUpgradeTestBase.java:233)
> 00:24:01 at
> org.apache.geode.cache.lucene.RollingUpgradeQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled.luceneQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled(RollingUpgradeQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRolled.java:91)
> 00:24:01
> 00:24:01 Caused by:
> 00:24:01 java.lang.reflect.InvocationTargetException
> 00:24:01 at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 00:24:01 at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 00:24:01 at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 00:24:01 at java.lang.reflect.Method.invoke(Method.java:566)
> 00:24:01 at
> org.apache.geode.cache.lucene.LuceneSearchWithRollingUpgradeTestBase.lambda$waitForRegionToHaveExpectedSize$0(LuceneSearchWithRollingUpgradeTestBase.java:250)
> 00:24:01 at
> org.awaitility.core.AssertionCondition.lambda$new$0(AssertionCondition.java:53)
> 00:24:01 at
> org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:248)
> 00:24:01 at
> org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:235)
> 00:24:01 at
> java.util.concurrent.FutureTask.run(FutureTask.java:264)
> 00:24:01 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> 00:24:01 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> 00:24:01 at java.lang.Thread.run(Thread.java:829)
> 00:24:01
> 00:24:01 Caused by:
> 00:24:01
> org.apache.geode.distributed.DistributedSystemDisconnectedException:
> Distribution manager on
> heavy-lifter-138290eb-3fbe-5801-8565-60d3c24284e9(143487)<v29>:42253 started
> at Fri Mar 25 07:23:21 UTC 2022: Member isn't responding to heartbeat
> requests, caused by org.apache.geode.ForcedDisconnectException: Member isn't
> responding to heartbeat requests
> 00:24:01 at
> org.apache.geode.distributed.internal.ClusterDistributionManager$Stopper.generateCancelledException(ClusterDistributionManager.java:2893)
> 00:24:01 at
> org.apache.geode.distributed.internal.InternalDistributedSystem$Stopper.generateCancelledException(InternalDistributedSystem.java:1186)
> 00:24:01 at
> org.apache.geode.internal.cache.GemFireCacheImpl$Stopper.generateCancelledException(GemFireCacheImpl.java:5212)
> 00:24:01 at
> org.apache.geode.CancelCriterion.checkCancelInProgress(CancelCriterion.java:83)
> 00:24:01 at
> org.apache.geode.internal.cache.GemFireCacheImpl.getRegion(GemFireCacheImpl.java:3327)
> 00:24:01 at
> org.apache.geode.internal.cache.GemFireCacheImpl.getRegion(GemFireCacheImpl.java:3193)
> 00:24:01 ... 12 more
> 00:24:01
> 00:24:01 Caused by:
> 00:24:01 org.apache.geode.ForcedDisconnectException:
> Member isn't responding to heartbeat requests
> 00:24:01 at
> org.apache.geode.distributed.internal.DistributionImpl$LifecycleListenerImpl.forcedDisconnect(DistributionImpl.java:941)
> 00:24:01 at
> org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.lambda$uncleanShutdownDS$0(GMSMembership.java:1792)
> 00:24:01 ... 1 more {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
