Re: ApplicationState Schema has drifted from DatabaseDescriptor

2011-02-09 Thread Brandon Williams
On Wed, Feb 9, 2011 at 4:31 PM, Aaron Morton wrote: > Thanks Gary. I'll keep an eye on things and see if it happens again. > > From reading the code I'm wondering if there is a small chance of a race > condition in HintedHandoffManager.waitForSchemaAgreement() . > > Could the following happen? I'm

Re: ApplicationState Schema has drifted from DatabaseDescriptor

2011-02-09 Thread Aaron Morton
Thanks Gary. I'll keep an eye on things and see if it happens again. From reading the code I'm wondering if there is a small chance of a race condition in HintedHandoffManager.waitForSchemaAgreement() .Could the following happen? I'm a little unsure on exactly how the endpoint state is removed from

Re: ApplicationState Schema has drifted from DatabaseDescriptor

2011-02-09 Thread Gary Dusbabek
Aaron, It looks like you're experiencing a side-effect of CASSANDRA-2083. There was at least one place (when node B received updated schema from node A) where gossip was not being updated with the correct schema even though DatabaseDescriptor had the right version. I'm pretty sure this is what yo

ApplicationState Schema has drifted from DatabaseDescriptor

2011-02-08 Thread Aaron Morton
I noticed this after I upgraded one node in a 0.7 cluster of 5 to the latest stable 0.7 build "2011-02-08_20-41-25" (upgraded  node was jb-cass1 below). This is a long email, you can jump to the end and help me out by checking something on your  07 cluster. This is the value from o.a.c.gms.FailureD