Re: puzzled why my cluster is slowing down

2013-01-07 Thread aaron morton
Can you slice up the "slows down" part a little more? Are you saying you are getting 4500 u-sec write latency ? Are you using secondary indexes? What sort of read queries are slowing down? What does the schema look like ? If the simple checks like CPU, iostat and GC logging in the cassandra l

puzzled why my cluster is slowing down

2013-01-07 Thread Brian Tarbox
I have a 4 node cluster with lots JVM memory and lots of system memory that slows down when I'm doing lots of writes. Running DataStax charts I see my read and write latency rise from 50-100 u-secs to 1500-4500 u-secs. This is across a 12 hour data load during which time the applied load is high