Re: Bottleneck for small inserts?

2017-06-15 Thread Eric Pederson
0.00 0.00 109.43 19023.30 2609.87 152186.40 8.09 273.89 14.30 0.01 23.64 dm-3 0.00 0.000.003.33 0.0026.67 8.00 0.000.25 0.07 0.02 -- Eric On Wed, Jun 14, 2017 at 11:17 PM, Eric Pederson wrote: > Using cassandra-stress with the out

Re: Bottleneck for small inserts?

2017-06-14 Thread Eric Pederson
o get past 36k rows/sec. The application schema has 64 columns (mostly ints) and the key is (date,sequence#). The standard stress schema has a lot fewer columns and no clustering column. Thanks, -- Eric On Wed, Jun 14, 2017 at 1:47 AM, Eric Pederson wrote: > Shoot - I didn't see

Re: Bottleneck for small inserts?

2017-06-13 Thread Eric Pederson
wrote: > Did you try adding more client stress nodes as Patrick recommended? > > On Tue, Jun 13, 2017 at 9:31 PM Eric Pederson wrote: > >> Scratch that theory - the flamegraphs show that GC is only 3-4% of two >> newer machine's overall processing, compared to 18% o

Re: Bottleneck for small inserts?

2017-06-13 Thread Eric Pederson
stats shows only blips of queueing. -- Eric On Mon, Jun 12, 2017 at 9:50 PM, Eric Pederson wrote: > Hi all - I wanted to follow up on this. I'm happy with the throughput > we're getting but I'm still curious about the bottleneck. > > The big thing that sticks ou

Re: Bottleneck for small inserts?

2017-06-12 Thread Eric Pederson
rcedelica.com/wordpress/wp-content/uploads/2017/05/flamegraph_ultva03_cars_batch2.svg> The slow node (ultva03) spends disproportional time in GC. Thanks, -- Eric On Thu, May 25, 2017 at 8:09 PM, Eric Pederson wrote: > Due to a cut and paste error those flamegraphs were a recording of the > whole sys

Re: Bottleneck for small inserts?

2017-05-25 Thread Eric Pederson
delica.com/wordpress/wp-content/uploads/2017/05/flamegraph_ultva03_cars_batch2.svg -- Eric On Thu, May 25, 2017 at 6:44 PM, Eric Pederson wrote: > Totally understood :) > > I forgot to mention - I set the /proc/irq/*/smp_affinity mask to include > all of the CPUs. Actually most of

Re: Bottleneck for small inserts?

2017-05-25 Thread Eric Pederson
raphs in a little bit - in the middle of > something and my brain doesn't multitask well :) > > On Thu, May 25, 2017 at 1:06 PM Eric Pederson wrote: > >> Hi Jonathan - >> >> It looks like these machines are configured to use CPU 0 for all I/O >> interrup

Re: Bottleneck for small inserts?

2017-05-25 Thread Eric Pederson
4-06-12/java-flame-graphs.html > > Are you tracking GC pauses? > > Jon > > On Mon, May 22, 2017 at 2:03 PM Eric Pederson wrote: > >> Hi all: >> >> I'm new to Cassandra and I'm doing some performance testing. One of >> things that I'm test

Bottleneck for small inserts?

2017-05-22 Thread Eric Pederson
Hi all: I'm new to Cassandra and I'm doing some performance testing. One of things that I'm testing is ingestion throughput. My server setup is: - 3 node cluster - SSD data (both commit log and sstables are on the same disk) - 64 GB RAM per server - 48 cores per server - Cassand