Re: Assassinate fails

2019-08-29 Thread Alain RODRIGUEZ
sometimes 3-5 times isn't enough. In such cases, shoot for >> the moon and try 20 assassination attempts in quick succession. >> >> What we are trying to do is to create a flood of messages requesting all >> nodes completely forget there used to be an entry within

Re: Assassinate fails

2019-08-16 Thread Alex
ssassination attempts in quick succession. >> >> What we are trying to do is to create a flood of messages requesting all >> nodes completely forget there used to be an entry within the gossip state >> for the given IP address. If each node can prune its own gossip sta

Re: Assassinate fails

2019-04-05 Thread Alain RODRIGUEZ
gt; deprecated node, the cosmetic issue will no longer be a concern in any > system.logs, nodetool describecluster commands, nor nodetool gossipinfo > output. > > > > > > -Original Message- > From: Kenneth Brotman [mailto:kenbrot...@yahoo.com.INVALID] > Sent: Th

RE: Assassinate fails

2019-04-04 Thread Kenneth Brotman
: Assassinate fails Alex, Did you remove the option JVM_OPTS="$JVM_OPTS -Dcassandra.replace_address=address_of_dead_node after the node started and then restart the node again? Are you sure there isn't a typo in the file? Ken -Original Message- From: Kenneth Brotman [mail

RE: Assassinate fails

2019-04-04 Thread Kenneth Brotman
om.INVALID] Sent: Thursday, April 04, 2019 10:31 AM To: user@cassandra.apache.org Subject: RE: Assassinate fails I see; system_auth is a separate keyspace. -Original Message- From: Jon Haddad [mailto:j...@jonhaddad.com] Sent: Thursday, April 04, 2019 10:17 AM To: user@cassandra.

RE: Assassinate fails

2019-04-04 Thread Kenneth Brotman
I see; system_auth is a separate keyspace. -Original Message- From: Jon Haddad [mailto:j...@jonhaddad.com] Sent: Thursday, April 04, 2019 10:17 AM To: user@cassandra.apache.org Subject: Re: Assassinate fails No, it can't. As Alain (and I) have said, since the system keyspa

Re: Assassinate fails

2019-04-04 Thread Jon Haddad
al Message- > From: Jon Haddad [mailto:j...@jonhaddad.com] > Sent: Thursday, April 04, 2019 9:52 AM > To: user@cassandra.apache.org > Subject: Re: Assassinate fails > > System != system_auth. > > On Thu, Apr 4, 2019 at 9:43 AM Kenneth Brotman > wrote: > > >

RE: Assassinate fails

2019-04-04 Thread Kenneth Brotman
Right, could be similar issue, same type of fix though. -Original Message- From: Jon Haddad [mailto:j...@jonhaddad.com] Sent: Thursday, April 04, 2019 9:52 AM To: user@cassandra.apache.org Subject: Re: Assassinate fails System != system_auth. On Thu, Apr 4, 2019 at 9:43 AM Kenneth

Re: Assassinate fails

2019-04-04 Thread Jon Haddad
> -Original Message- > From: Jon Haddad [mailto:j...@jonhaddad.com] > Sent: Thursday, April 04, 2019 9:21 AM > To: user@cassandra.apache.org > Subject: Re: Assassinate fails > > Ken, > > Alain is right about the system tables. What you're describing only > w

Re: Assassinate fails

2019-04-04 Thread Alex
# wait for this to return before moving > on to the next node. > > Restart them all using this method, then run nodetool status again and see if > it is listed. > > Once other thing, I recall you said something about having to terminate a > node and

RE: Assassinate fails

2019-04-04 Thread Kenneth Brotman
should never use the default cassandra user. -Original Message- From: Jon Haddad [mailto:j...@jonhaddad.com] Sent: Thursday, April 04, 2019 9:21 AM To: user@cassandra.apache.org Subject: Re: Assassinate fails Ken, Alain is right about the system tables. What you're describing

Re: Assassinate fails

2019-04-04 Thread Alex
ait for this to return before moving > on to the next node. > > Restart them all using this method, then run nodetool status again and see if > it is listed. > > Once other thing, I recall you said something about having to terminate a > node and then replace it. Make su

Re: Assassinate fails

2019-04-04 Thread Jon Haddad
From: Alain RODRIGUEZ [mailto:arodr...@gmail.com] > Sent: Thursday, April 04, 2019 8:46 AM > To: user cassandra.apache.org > Subject: Re: Assassinate fails > > > > Hi Alex, > > > > About previous advices: > > > > You might have inconsistent data in your syst

RE: Assassinate fails

2019-04-04 Thread Kenneth Brotman
The trick below I got from the book Mastering Cassandra. You have to set the consistency to ALL for it to work. I thought you guys knew that one. From: Alain RODRIGUEZ [mailto:arodr...@gmail.com] Sent: Thursday, April 04, 2019 8:46 AM To: user cassandra.apache.org Subject: Re: Assassinate

Re: Assassinate fails

2019-04-04 Thread Alain RODRIGUEZ
omething about having to terminate a > node and then replace it. Make sure that whichever node you did the > –Dreplace flag on, does not still have it set when you start cassandra on > it again! > > > > *From:* Alex [mailto:m...@aca-o.com] > *Sent:* Thursday, April 04, 20

Re: Assassinate fails

2019-04-04 Thread Jeff Jirsa
t; Sent: Thursday, April 04, 2019 4:58 AM > To: user@cassandra.apache.org > Subject: Re: Assassinate fails > > Hi Anthony, > > Thanks for your help. > > I tried to run multiple times in quick succession but it fails with : > > -- StackTrace -- > java.lang

RE: Assassinate fails

2019-04-04 Thread Nick Hatfield
again! From: Alex [mailto:m...@aca-o.com] Sent: Thursday, April 04, 2019 4:58 AM To: user@cassandra.apache.org Subject: Re: Assassinate fails Hi Anthony, Thanks for your help. I tried to run multiple times in quick succession but it fails with : -- StackTrace -- java.lang.RuntimeException: En

RE: Assassinate fails

2019-04-04 Thread Kenneth Brotman
Subject: Re: Assassinate fails Hi Anthony, Thanks for your help. I tried to run multiple times in quick succession but it fails with : -- StackTrace -- java.lang.RuntimeException: Endpoint still alive: /192.168.1.18 generation changed while trying to assassinate it at

Re: Assassinate fails

2019-04-04 Thread Alex
ar ./jmxterm.jar >>> >>> $>open localhost:7199 >>> >>> $>bean org.apache.cassandra.net:type=Gossiper >>> >>> $>run unsafeAssassinateEndpoint 192.168.1.18 >>> >>> $>quit >>> >>> >>> Happy de

Re: Assassinate fails

2019-04-03 Thread Anthony Grasso
> > $>quit > > > > > > Happy deleting > > > > -Original Message- > > From: Alex [mailto:m...@aca-o.com] > > Sent: Wednesday, April 03, 2019 10:42 AM > > To: user@cassandra.apache.org > > Subject: Assassinate fails > > > &

Re: Assassinate fails

2019-04-03 Thread Alex
pril 03, 2019 10:42 AM To: user@cassandra.apache.org Subject: Assassinate fails Hello, Short story: - I had to replace a dead node in my cluster - 1 week after, dead node is still seen as DN by 3 out of 5 nodes - dead node has null host_id - assassinate on dead node fails with error How can I get

RE: Assassinate fails

2019-04-03 Thread Nick Hatfield
.168.1.18 $>quit Happy deleting -Original Message- From: Alex [mailto:m...@aca-o.com] Sent: Wednesday, April 03, 2019 10:42 AM To: user@cassandra.apache.org Subject: Assassinate fails Hello, Short story: - I had to replace a dead node in my cluster - 1 week after, dead node is still see

Assassinate fails

2019-04-03 Thread Alex
Hello, Short story: - I had to replace a dead node in my cluster - 1 week after, dead node is still seen as DN by 3 out of 5 nodes - dead node has null host_id - assassinate on dead node fails with error How can I get rid of this dead node ? Long story: I had a 3 nodes cluster (Cassandra 3.9)