Re: pycassa timeouts resolved by killing a random node in the ring

2011-04-12 Thread aaron morton
First, lets check if the timeouts are client or server side. What was the timeout error stack ? Were they (python/thrift) socket timeouts or TimedOutException's raised by the cassandra thrift code. Is it across all requests / clients or say just read? Have you tried asking on http://groups.go

pycassa timeouts resolved by killing a random node in the ring

2011-04-12 Thread Jason Harvey
Interesting issue this morning. My apps started throwing a bunch of pycassa timeouts all of a sudden. The ring looked perfect. No load issues anywhere, and no errors in the logs. The site was basically down, so I got desperate and whacked a random node in the ring. As soon as gossip saw it go dow