Re: Damaged commit log disk causes Cassandra client to get stuck

2011-08-02 Thread Jim Ancona
e took down the Cassandra node with the faulty >> commit log disk, the client continued to write and didn't seem to bother >> (which is what we expected to happen in the first place, but didn't). >> >> From: aaron morton [mailto:aa...@thelastpickle.com] >> Sent: Monday

Re: Damaged commit log disk causes Cassandra client to get stuck

2011-08-02 Thread Jim Ancona
And to add to that – when we took down the Cassandra node with the faulty > commit log disk, the client continued to write and didn't seem to bother > (which is what we expected to happen in the first place, but didn't). > > From: aaron morton [mailto:aa...@thelastpickle.com] &

Re: Damaged commit log disk causes Cassandra client to get stuck

2011-07-31 Thread aaron morton
th the faulty > commit log disk, the client continued to write and didn't seem to bother > (which is what we expected to happen in the first place, but didn't). > > From: aaron morton [mailto:aa...@thelastpickle.com] > Sent: Monday, August 01, 2011 12:19 AM > To:

RE: Damaged commit log disk causes Cassandra client to get stuck

2011-07-31 Thread Lior Golan
Monday, August 01, 2011 12:19 AM To: user@cassandra.apache.org Subject: Re: Damaged commit log disk causes Cassandra client to get stuck A couple of timeouts should have kicked in. First the rpc_timeout on the server side should have kicked in and given the client a (thrift) TimedOutException. Se

Re: Damaged commit log disk causes Cassandra client to get stuck

2011-07-31 Thread aaron morton
A couple of timeouts should have kicked in. First the rpc_timeout on the server side should have kicked in and given the client a (thrift) TimedOutException. Secondly a client side socket timeout should be set so the client will timeout the socket. Did either of these appear in the client side

Damaged commit log disk causes Cassandra client to get stuck

2011-07-31 Thread Lior Golan
In one of our test clusters we had a damaged commit log disks in one of the nodes. We have replication factor = 2 in this cluster, and write with consistency level = ONE. So we expected writes will not be affected by such an issue. But what actually happened is that the client that was writing