[
https://issues.apache.org/jira/browse/GEODE-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Schuchardt updated GEODE-6522:
------------------------------------
Summary: Server hangs during shutdown after becoming membership coordinator
(was: If a member is deposed as membership coordinator it never becomes one
again)
> Server hangs during shutdown after becoming membership coordinator
> ------------------------------------------------------------------
>
> Key: GEODE-6522
> URL: https://issues.apache.org/jira/browse/GEODE-6522
> Project: Geode
> Issue Type: Bug
> Components: membership
> Reporter: Bruce Schuchardt
> Assignee: Bruce Schuchardt
> Priority: Major
>
> The fix for GEODE-870 introduced a mechanism to shut down the View Creator
> thread if a member is no longer the coordinator. This seems to be messing up
> - the view creator is marking itself to exit when sending a new view,
> thinking for some reason it's no longer the coordinator in the installView
> method.
> {noformat}
> [info 2019/03/13 08:38:24.959 PDT <Geode Membership View Creator> tid=0x147]
> finished waiting for responses to view preparation
> [info 2019/03/13 08:38:24.959 PDT <Geode Membership View Creator> tid=0x147]
> received new view:
> View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|15] members:
> [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown:
> [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000,
> turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003,
> turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005]
> old view is:
> View[turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000|3]
> members: [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000,
> turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003,
> turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023]
> [info 2019/03/13 08:38:24.974 PDT <Geode Membership View Creator> tid=0x147]
> Failure detection is now watching
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015; suspects are
> {turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002=View[turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000|3]
> members:
> [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000,
> turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003,
> turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023]}
> [info 2019/03/13 08:38:24.981 PDT <Geode Membership View Creator> tid=0x147]
> sending new view
> View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|15] members:
> [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown:
> [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000,
> turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003,
> turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005]
> [info 2019/03/13 08:38:24.981 PDT <Geode Membership View Creator> tid=0x147]
> BRUCE: setting shutdown flag in view creator
> java.lang.Exception: stack trace
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.setShutdownFlag(GMSJoinLeave.java:2247)
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2713)
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.sendInitialView(GMSJoinLeave.java:2220)
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2299)
> [info 2019/03/13 08:38:24.982 PDT <Geode Membership View Creator> tid=0x147]
> View Creator thread is exiting
> [info 2019/03/13 08:38:24.982 PDT <Geode Membership View Creator> tid=0x147]
> BRUCE: setting shutdown flag in view creator
> java.lang.Exception: stack trace
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.setShutdownFlag(GMSJoinLeave.java:2247)
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2379)
> [info 2019/03/13 08:38:26.416 PDT <vm_4_thr_8_bridge_2_1_host1_17023>
> tid=0x150] GemFireCache[id = 66144348; isClosing = true; isShutDownAll =
> false; created = Wed Mar 13 08:34:41 PDT 2019; server = false; copyOnRead =
> false; lockLease = 120; lockTimeout = 60]: Now closing.
> ...
> [info 2019/03/13 08:38:27.495 PDT <Geode Membership View Creator> tid=0x161]
> View Creator thread is starting
> [info 2019/03/13 08:38:27.507 PDT <Geode Membership View Creator> tid=0x161]
> preparing new view
> View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|21] members:
> [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown:
> [turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019]
> [info 2019/03/13 08:38:27.508 PDT <Geode Membership View Creator> tid=0x161]
> finished waiting for responses to view preparation
> [info 2019/03/13 08:38:27.508 PDT <Geode Membership View Creator> tid=0x161]
> received new view:
> View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|21] members:
> [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown:
> [turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019]
> old view is:
> View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|15] members:
> [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown:
> [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000,
> turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003,
> turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005]
> [info 2019/03/13 08:38:27.566 PDT <Geode Membership View Creator> tid=0x161]
> sending new view
> View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|21] members:
> [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002,
> turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead},
> turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015,
> turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown:
> [turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019]
> [info 2019/03/13 08:38:27.567 PDT <Geode Membership View Creator> tid=0x161]
> BRUCE: setting shutdown flag in view creator
> java.lang.Exception: stack trace
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.setShutdownFlag(GMSJoinLeave.java:2247)
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2713)
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.sendInitialView(GMSJoinLeave.java:2220)
> at
> org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2299)
> [info 2019/03/13 08:38:27.567 PDT <Geode Membership View Creator> tid=0x161]
> View Creator thread is exiting
> {noformat}
> etc.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
