Re: Consistent read timeouts for bursts of reads

2016-03-04 Thread Mike Heffner
Emils, We believe we've tracked it down to the following issue: https://issues.apache.org/jira/browse/CASSANDRA-11302, introduced in 2.1.5. We are running a build of 2.2.5 with that patch and so far have not seen any more timeouts. Mike On Fri, Mar 4, 2016 at 3:14 AM, Emīls Šolmanis wrote: >

Re: Consistent read timeouts for bursts of reads

2016-03-04 Thread Emīls Šolmanis
Mike, Is that where you've bisected it to having been introduced? I'll see what I can do, but doubt it, since we've long since upgraded prod to 2.2.4 (and stage before that) and the tests I'm running were for a new feature. On Fri, 4 Mar 2016 03:54 Mike Heffner, wrote: > Emils, > > I realize t

Re: Consistent read timeouts for bursts of reads

2016-03-03 Thread Mike Heffner
Emils, I realize this may be a big downgrade, but are you timeouts reproducible under Cassandra 2.1.4? Mike On Thu, Feb 25, 2016 at 10:34 AM, Emīls Šolmanis wrote: > Having had a read through the archives, I missed this at first, but this > seems to be *exactly* like what we're experiencing. >

Re: Consistent read timeouts for bursts of reads

2016-03-01 Thread Carlos Alonso
We have had similar issues sometimes. Usually the problem was that failing queries where reading the same partition that another query still running and that partition is too big. The fact that is reading the same partition is why your query works upon retry. The fact that the partition (or the r

Re: Consistent read timeouts for bursts of reads

2016-02-25 Thread Emīls Šolmanis
Having had a read through the archives, I missed this at first, but this seems to be *exactly* like what we're experiencing. http://www.mail-archive.com/user@cassandra.apache.org/msg46064.html Only difference is we're getting this for reads and using CQL, but the behaviour is identical. On Thu,

Consistent read timeouts for bursts of reads

2016-02-25 Thread Emīls Šolmanis
Hello, We're having a problem with concurrent requests. It seems that whenever we try resolving more than ~ 15 queries at the same time, one or two get a read timeout and then succeed on a retry. We're running Cassandra 2.2.4 accessed via the 2.1.9 Datastax driver on AWS. What we've found while