Re: Slow reads on C* 2.0.15 using Spark Cassandra

2015-06-29 Thread Nathan Bijnens
One more update, it looks like the driver is generating this CQL statements: SELECT "test_id", "channel", "ts", "event", "groups" FROM "KEYSPACE"."test" WHERE token("test_id") > ? AND token("test_id") <= ? ALLOW FILTERING; Best regards, Nathan On Fri, Jun 26, 2015 at 8:16 PM Nathan Bijnens

Re: Slow reads on C* 2.0.15 using Spark Cassandra

2015-06-26 Thread Nathan Bijnens
Thanks for the suggestion, will take a look. Our code looks like this: val rdd = sc.cassandraTable[EventV0](keyspace, "test") val transformed = rdd.map{e => EventV1(e.testId, e.ts, e.channel, e.groups, e.event)} transformed.saveToCassandra(keyspace, "test_v1") Not sure if this code might transl

Re: Slow reads on C* 2.0.15 using Spark Cassandra

2015-06-26 Thread Nate McCall
> We notice incredibly slow reads, 600mb in an hour, we are using quorum LOCAL_ONE reads. > The load_one of Cassandra increases from <1 to 60! There is no CPU wait, only user & nice. Without seeing the code and query, it's hard to tell, but I noticed something similar when we had a client incorrec

Slow reads on C* 2.0.15 using Spark Cassandra

2015-06-26 Thread Nathan Bijnens
We are using the Spark Cassandra driver, version 1.2.0 (Spark 1.2.1) connecting to a 6 node bare metal (16gb ram, Xeon E3-1270 (8core), 4x 7,2k SATA disks) Cassandra cluster. Spark runs on a separate Mesos cluster. We are running a transformation job, where we read the complete contents of a table