[ https://issues.apache.org/jira/browse/GEODE-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17440625#comment-17440625 ]
Kamilla Aslami edited comment on GEODE-9739 at 11/8/21, 5:33 PM: ----------------------------------------------------------------- Removing membership component because this issue seems to be caused by the "create gateway-sender" command and is not membership-related. Here's my analysis: This test creates 2 WAN sites: one with three 1.12 members vm0, vm1, vm2 and another with three 1.15 members: vm4, vm5, and vm6. Then it upgrades vm0 and vm2 to 1.15 and attempts to create a gateway sender: {noformat} [vm0] [info 2021/10/13 23:34:21.093 UTC <RMI TCP Connection(3)-10.0.0.147> tid=0x62] Executing command: create gateway-sender --id=toSite2 --remote-distributed-system-id=1{noformat} This command is supposed to fail, and I see the expected error message in the logs ~40 seconds later (usually it fails without delay): {noformat} [vm4] [info 2021/10/13 23:35:00.906 UTC <ReconnectThread> tid=0x37d] Shutting down DistributionManager heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919. At least one Exception occurred. Command result for <create gateway-sender --id=toSite2 --remote-distributed-system-id=1>: Gateway Sender cannot be created until all members are the current version{noformat} It looks like the "create gateway-sender" command hung because there were no logs for at least 15 seconds after it was executed. ~25 seconds after the command was executed, vm4, vm5, and vm6 started having problems connecting to each other: {noformat} [vm6] [info 2021/10/13 23:34:47.076 UTC <Geode Failure Detection thread 2> tid=0x35f] received suspect message from myself for heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919: Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.083 UTC <Geode Failure Detection thread 2> tid=0x35f] All other members are suspect at this point [vm6] [info 2021/10/13 23:34:47.134 UTC <Geode Failure Detection thread 3> tid=0x360] received suspect message from myself for heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920: Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.787 UTC <Geode Failure Detection thread 2> tid=0x35f] Performing availability check for suspect member heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919 reason=Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.809 UTC <Geode Failure Detection thread 2> tid=0x35f] All other members are suspect at this point [vm6] [info 2021/10/13 23:34:47.844 UTC <Geode Failure Detection thread 3> tid=0x360] All other members are suspect at this point [vm6] [info 2021/10/13 23:34:47.866 UTC <Geode Failure Detection thread 4> tid=0x362] Performing availability check for suspect member heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920 reason=Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.866 UTC <Geode Failure Detection thread 4> tid=0x362] All other members are suspect at this point ...{noformat} Most of the availability checks passed, but one member got kicked out, and the quorum was lost: {noformat} [vm4] [info 2021/10/13 23:34:54.355 UTC <Geode Membership View Creator> tid=0x35a] View Creator is processing 1 requests for the next membership view ([RemoveMemberMessage(heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920; reason=Member isn't responding to heartbeat requests)]) [vm4] [info 2021/10/13 23:34:54.486 UTC <Geode Membership View Creator> tid=0x35a] heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920 had a weight of 15 [vm4] [warn 2021/10/13 23:34:54.517 UTC <Geode Membership View Creator> tid=0x35a] total weight lost in this view change is 15 of 28. Quorum has been lost! [vm4] [fatal 2021/10/13 23:34:54.989 UTC <Geode Membership View Creator> tid=0x35a] Possible loss of quorum due to the loss of 1 cache processes: [heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920]{noformat} vm4, vm5, and vm6 also reported that heartbeat-generation thread overslept by more than a full period: {noformat} [vm6] [warn 2021/10/13 23:34:46.402 UTC <Geode Heartbeat Sender> tid=0x343] Failure detection heartbeat-generation thread overslept by more than a full period. Asleep time: 7,274,351,195 nanoseconds. Period: 2,500,000,000 nanoseconds. [vm4] [warn 2021/10/13 23:34:49.469 UTC <Geode Heartbeat Sender> tid=0x358] Failure detection heartbeat-generation thread overslept by more than a full period. Asleep time: 19,549,608,016 nanoseconds. Period: 2,500,000,000 nanoseconds. [vm5] [warn 2021/10/13 23:34:52.864 UTC <Geode Heartbeat Sender> tid=0x338] Failure detection heartbeat-generation thread overslept by more than a full period. Asleep time: 5,271,931,427 nanoseconds. Period: 2,500,000,000 nanoseconds.{noformat} The "heartbeat-generation thread overslept" message implies that there was a resource issue. In this case, it might've been caused by the create gateway-sender command as it took 40 seconds to complete. I don't see membership-related issues in the logs; the membership component operated correctly based upon the analysis. I ran this test 200 times but couldn't reproduce the issue. was (Author: kaslami): Removing membership component because this issue seems to be caused by the "create gateway-sender" command and is not membership-related. Here's my analysis: This test creates 2 WAN sites: one with three 1.12 members vm0, vm1, vm2 and another with three 1.15 members: vm4, vm5, and vm6. Then it upgrades vm0 and vm2 to 1.15 and attempts to create a gateway sender: {noformat} [vm0] [info 2021/10/13 23:34:21.093 UTC <RMI TCP Connection(3)-10.0.0.147> tid=0x62] Executing command: create gateway-sender --id=toSite2 --remote-distributed-system-id=1{noformat} This command is supposed to fail, and I see the expected error message in the logs ~40 seconds later (usually it fails without delay): {noformat} [vm4] [info 2021/10/13 23:35:00.906 UTC <ReconnectThread> tid=0x37d] Shutting down DistributionManager heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919. At least one Exception occurred. Command result for <create gateway-sender --id=toSite2 --remote-distributed-system-id=1>: Gateway Sender cannot be created until all members are the current version{noformat} It looks like the "create gateway-sender" command hung because there were no logs for at least 15 seconds after it was executed. ~25 seconds after the command was executed, vm4, vm5, and vm6 started having problems connecting to each other: {noformat} [vm6] [info 2021/10/13 23:34:47.076 UTC <Geode Failure Detection thread 2> tid=0x35f] received suspect message from myself for heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919: Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.083 UTC <Geode Failure Detection thread 2> tid=0x35f] All other members are suspect at this point [vm6] [info 2021/10/13 23:34:47.134 UTC <Geode Failure Detection thread 3> tid=0x360] received suspect message from myself for heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920: Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.787 UTC <Geode Failure Detection thread 2> tid=0x35f] Performing availability check for suspect member heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919 reason=Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.809 UTC <Geode Failure Detection thread 2> tid=0x35f] All other members are suspect at this point [vm6] [info 2021/10/13 23:34:47.844 UTC <Geode Failure Detection thread 3> tid=0x360] All other members are suspect at this point [vm6] [info 2021/10/13 23:34:47.866 UTC <Geode Failure Detection thread 4> tid=0x362] Performing availability check for suspect member heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920 reason=Member isn't responding to heartbeat requests [vm6] [info 2021/10/13 23:34:47.866 UTC <Geode Failure Detection thread 4> tid=0x362] All other members are suspect at this point ...{noformat} Most of the availability checks passed, but one member got kicked out, and the quorum was lost: {noformat} [vm4] [info 2021/10/13 23:34:54.355 UTC <Geode Membership View Creator> tid=0x35a] View Creator is processing 1 requests for the next membership view ([RemoveMemberMessage(heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920; reason=Member isn't responding to heartbeat requests)]) [vm4] [info 2021/10/13 23:34:54.486 UTC <Geode Membership View Creator> tid=0x35a] heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920 had a weight of 15 [vm4] [warn 2021/10/13 23:34:54.517 UTC <Geode Membership View Creator> tid=0x35a] total weight lost in this view change is 15 of 28. Quorum has been lost! [vm4] [fatal 2021/10/13 23:34:54.989 UTC <Geode Membership View Creator> tid=0x35a] Possible loss of quorum due to the loss of 1 cache processes: [heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920]{noformat} vm4, vm5, and vm6 also reported that heartbeat-generation thread overslept by more than a full period: {noformat} [vm6] [warn 2021/10/13 23:34:46.402 UTC <Geode Heartbeat Sender> tid=0x343] Failure detection heartbeat-generation thread overslept by more than a full period. Asleep time: 7,274,351,195 nanoseconds. Period: 2,500,000,000 nanoseconds. [vm4] [warn 2021/10/13 23:34:49.469 UTC <Geode Heartbeat Sender> tid=0x358] Failure detection heartbeat-generation thread overslept by more than a full period. Asleep time: 19,549,608,016 nanoseconds. Period: 2,500,000,000 nanoseconds. [vm5] [warn 2021/10/13 23:34:52.864 UTC <Geode Heartbeat Sender> tid=0x338] Failure detection heartbeat-generation thread overslept by more than a full period. Asleep time: 5,271,931,427 nanoseconds. Period: 2,500,000,000 nanoseconds.{noformat} The "heartbeat-generation thread overslept" message implies that there was a resource issue. In this case, it might've been caused by the create gateway-sender command as it took 40 seconds to complete. I don't see membership-related issues in the logs; the membership component operated correctly based upon the analysis. > CI: WANRollingUpgradeCreateGatewaySenderMixedSiteOneCurrentSiteTwo failed > with MemberDisconnectedException > ---------------------------------------------------------------------------------------------------------- > > Key: GEODE-9739 > URL: https://issues.apache.org/jira/browse/GEODE-9739 > Project: Geode > Issue Type: Bug > Components: wan > Affects Versions: 1.15.0 > Reporter: Kamilla Aslami > Priority: Major > Labels: needsTriage > > {noformat} > WANRollingUpgradeCreateGatewaySenderMixedSiteOneCurrentSiteTwo > > CreateGatewaySenderMixedSiteOneCurrentSiteTwo[from_v1.12.2] FAILED > java.lang.AssertionError: Suspicious strings were written to the log > during this run. > Fix the strings or use IgnoredException.addIgnoredException to ignore. > ----------------------------------------------------------------------- > Found suspect string in 'dunit_suspect-vm6.log' at line 481 [fatal > 2021/10/13 23:34:55.115 UTC <unicast > receiver,heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3-37210> tid=830] > Membership service failure: Membership coordinator > heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919 > has declared that a network partition has occurred > > org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: > Membership coordinator > heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(39325:locator)<ec><v0>:53919 > has declared that a network partition has occurred > at > org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1807) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.processNetworkPartitionMessage(GMSJoinLeave.java:1466) > at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1367) > at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1303) > at org.jgroups.JChannel.invokeCallback(JChannel.java:816) > at org.jgroups.JChannel.up(JChannel.java:741) > at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030) > at org.jgroups.protocols.FRAG2.up(FRAG2.java:165) > at org.jgroups.protocols.FlowControl.up(FlowControl.java:390) > at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1077) > at > org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:792) > at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433) > at > org.apache.geode.distributed.internal.membership.gms.messenger.StatRecorder.up(StatRecorder.java:72) > at > org.apache.geode.distributed.internal.membership.gms.messenger.AddressManager.up(AddressManager.java:70) > at org.jgroups.protocols.TP.passMessageUp(TP.java:1658) > at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1876) > at org.jgroups.util.DirectExecutor.execute(DirectExecutor.java:10) > at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1789) > at org.jgroups.protocols.TP.receive(TP.java:1714) > at > org.apache.geode.distributed.internal.membership.gms.messenger.Transport.receive(Transport.java:160) > at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:701) > at java.base/java.lang.Thread.run(Thread.java:829) > ----------------------------------------------------------------------- > Found suspect string in 'dunit_suspect-vm4.log' at line 549 [fatal > 2021/10/13 23:34:54.989 UTC <Geode Membership View Creator> tid=858] Possible > loss of quorum due to the loss of 1 cache processes: > [heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920] > ----------------------------------------------------------------------- > Found suspect string in 'dunit_suspect-vm4.log' at line 551 [fatal > 2021/10/13 23:34:56.179 UTC <Geode Membership View Creator> tid=858] > Membership service failure: Exiting due to possible network partition event > due to loss of 1 cache processes: > [heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920] > > org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: > Exiting due to possible network partition event due to loss of 1 cache > processes: > [heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3(41092)<v1>:53920] > at > org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1807) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.access$1300(GMSJoinLeave.java:80) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2588) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.createAndSendView(GMSJoinLeave.java:2565) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2340) > ----------------------------------------------------------------------- > Found suspect string in 'dunit_suspect-vm5.log' at line 466 [fatal > 2021/10/13 23:34:55.919 UTC <unicast > receiver,heavy-lifter-f2dde05e-a413-5672-b192-2396306457b3-34580> tid=819] > Membership service failure: Member isn't responding to heartbeat requests > > org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: > Member isn't responding to heartbeat requests > at > org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1807) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122) > at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.processRemoveMemberMessage(GMSJoinLeave.java:725) > at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1367) > at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1303) > at org.jgroups.JChannel.invokeCallback(JChannel.java:816) > at org.jgroups.JChannel.up(JChannel.java:741) > at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030) > at org.jgroups.protocols.FRAG2.up(FRAG2.java:165) > at org.jgroups.protocols.FlowControl.up(FlowControl.java:390) > at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1077) > at > org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:792) > at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433) > at > org.apache.geode.distributed.internal.membership.gms.messenger.StatRecorder.up(StatRecorder.java:72) > at > org.apache.geode.distributed.internal.membership.gms.messenger.AddressManager.up(AddressManager.java:70) > at org.jgroups.protocols.TP.passMessageUp(TP.java:1658) > at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1876) > at org.jgroups.util.DirectExecutor.execute(DirectExecutor.java:10) > at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1789) > at org.jgroups.protocols.TP.receive(TP.java:1714) > at > org.apache.geode.distributed.internal.membership.gms.messenger.Transport.receive(Transport.java:160) > at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:701) > at java.base/java.lang.Thread.run(Thread.java:829) > at org.junit.Assert.fail(Assert.java:89) > at > org.apache.geode.test.dunit.internal.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:420) > at > org.apache.geode.test.dunit.internal.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:436) > at > org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.cleanupAllVms(JUnit4DistributedTestCase.java:551) > at > org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.doTearDownDistributedTestCase(JUnit4DistributedTestCase.java:498) > at > org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.tearDownDistributedTestCase(JUnit4DistributedTestCase.java:481) > at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:566) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at > org.apache.geode.test.junit.rules.DescribedExternalResource$1.evaluate(DescribedExternalResource.java:40) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > at org.junit.runners.ParentRunner.run(ParentRunner.java:413) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:27) > at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at org.junit.runners.ParentRunner.run(ParentRunner.java:413) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at org.junit.runner.JUnitCore.run(JUnitCore.java:115) > at > org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at > java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) > at java.util.Iterator.forEachRemaining(Iterator.java:133) > at > java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) > at > java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at > java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) > at > org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:82) > at > org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:73) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52) > at > org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96) > at > org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75) > at > org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99) > at > org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79) > at > org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61) > at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:566) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) > at com.sun.proxy.$Proxy2.stop(Unknown Source) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133) > at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:566) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182) > at > org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164) > at > org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414) > at > org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) > at > org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at > org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56) > at java.lang.Thread.run(Thread.java:829) > {noformat} -- This message was sent by Atlassian Jira (v8.20.1#820001)