I restarted 48 nodes and every one came up fine. I was just wondering why the status run on the restarted node has no ID until it has finished dealing with whatever it does when starting up but it shows up immediately when status is run on any other node.
I guess it prompts the question: how much time should elapse between restarting each node? It seems to be something <60 seconds but I suppose it would depend on whatever was lingering in the commit directory. -----Original Message----- From: Bowen Song via user <user@cassandra.apache.org> Sent: Friday, September 23, 2022 3:47 PM To: user@cassandra.apache.org Subject: Re: Restart Cassandra EXTERNAL Did the node finish starting when you checked the "nodetool status" output? Try "nodetool netstats" on the starting node, the output will show "Mode: NORMAL" if it has finished starting. It's also worth checking the "nodetool info" output, and make sure "Gossip active" and "Native Transport active" (unless you have disabled it) are "true". On 23/09/2022 08:17, Marc Hoppins wrote: > Hi all, > > Restarting the service on a node. Checking status from a remote node, I see: > > (prod) marc.hoppins.ipa@ba-cassandra01:~ $ /opt/cassandra/bin/nodetool status > -r|grep 03 > UN ba-cassandra09 779.03 GiB 16 ? > 1fc8061d-2dd4-4b2c-97fa-e492063da495 SSW09 > UN ba-cassandra20 796.94 GiB 16 ? > c6b43e76-bd5d-4672-a62a-83a06030578d SSW09 > UN ba-cassandra10 750.84 GiB 16 ? > c03ae9c6-89cb-4e65-a1ef-a56e2efc24da SSW09 > UN ba-cassandra04 785.97 GiB 16 ? > 16dac20f-89fe-435c-8b49-d80a03fe239e SSW09 > DN ba-cassandra03 729.43 GiB 16 ? > 8785b173-6b68-45a4-ad38-e9b4036ffaf5 SSW09 > UN dr1-cassandra18 738.9 GiB 16 ? > 84348044-b6c6-44d0-9038-b1d49d39e496 SSW02 > UN dr1-cassandra03 783.04 GiB 16 ? > 21dac8e4-b556-48f1-873d-fb2876e2c349 SSW02 > > But when checking locally, I see: > > ?N ba-cassandra08 ? 16 ? > 8520bdcd-1cfb-431f-a99c-15b8ca288e96 SSW09 > ?N ba-cassandra04 ? 16 ? > 16dac20f-89fe-435c-8b49-d80a03fe239e SSW09 > ?N ba-cassandra11 ? 16 ? > cf010de0-657c-4135-beec-7ba37cc3d8f4 SSW09 > ?N ba-cassandra18 ? 16 ? > 994c67d7-e6f9-4419-a02b-b5296ec92cb0 SSW09 > UN ba-cassandra03 146.03 GiB 16 ? > SSW09 > ?N ba-cassandra21 ? 16 ? > 7a8fc8c4-fb64-4bb7-ad5c-cb5112d9f783 SSW09 > ?N ba-cassandra13 ? 16 ? > 853f095a-c780-473d-85f0-b8d047d745f1 SSW09 > > When I recheck it, I notice that the data count increases to the correct > amount after some small time, the node ID appears for the local status, and > the remote status shows as UN. If the remote status shows the node ID, why > is it missing locally? Is the node ID only stored on the seeds?
