Re: Attached profiled data but need help understanding it

Hi Kant, You'll find more information about ixgbevf here  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sriov-networking.htmlI repeat myself but don't underestimate VMs placement: same AZ? same placement group? etc.Note that LWT are not discouraged but as the doc says: "[...] reserve lightw

Re: Attached profiled data but need help understanding it

Hi Romain, We may be able to achieve what we need without LWT but that would require bunch of changes from the application side and possibly introducing caching layers and designing solution around that. But for now, we are constrained to use LWT's for another month or so. All said, I still would

Re: Attached profiled data but need help understanding it

Also, I should have mentioned that it would be a good idea to spawn your three benchmark instances in the same AZ, then try with one instance on each AZ to see how network latency affects your LWT rate. The lower latency is achievable with three instances on the same placement group of course bu

Re: Attached profiled data but need help understanding it

Hi Kant, > By backporting you mean I should cherry pick CASSANDRA-11966 commit and > compile from source? Yes Regarding the network utilization: you checked throughput but latency is more important for LWT. That's why you should make sure your m4 instances (both C* and client) are using ixgbevf

Re: Attached profiled data but need help understanding it

Hi Romain, I am using Cassandra version 3.0.9 and here is the generated report (Graphical view) of my thread dump as well!. Just send this over in case if it helps. Thanks, kant On Tue, Feb

Re: Attached profiled data but need help understanding it

Hi Romain, Thanks again. My response are inline. kant On Tue, Feb 28, 2017 at 10:04 AM, Romain Hardouin wrote: > > we are currently using 3.0.9. should we use 3.8 or 3.10 > > No, don't use 3.X in production unless you really need a major feature. > I would advise to stick to 3.0.X (i.e. 3.0.1

Re: Attached profiled data but need help understanding it

> we are currently using 3.0.9.  should we use 3.8 or 3.10 No, don't use 3.X in production unless you really need a major feature.I would advise to stick to 3.0.X (i.e. 3.0.11 now).You can backport CASSANDRA-11966 easily but of course you have to deploy from source as a prerequisite. > I haven't

Re: Attached profiled data but need help understanding it

Hi! My answers are inline. On Mon, Feb 27, 2017 at 11:48 AM, Kant Kodali wrote: > > > On Mon, Feb 27, 2017 at 10:30 AM, Romain Hardouin > wrote: > >> Hi, >> >> Regarding shared pool workers see CASSANDRA-11966. You may have to >> backport it depending on your Cassandra version. >> > > *we a

Re: Attached profiled data but need help understanding it

On Mon, Feb 27, 2017 at 10:30 AM, Romain Hardouin wrote: > Hi, > > Regarding shared pool workers see CASSANDRA-11966. You may have to > backport it depending on your Cassandra version. > *we are currently using 3.0.9. should we use 3.8 or 3.10?* > > Did you try to lower compaction throughp

Re: Attached profiled data but need help understanding it

Hi, Regarding shared pool workers see CASSANDRA-11966. You may have to backport it depending on your Cassandra version.  Did you try to lower compaction throughput to see if it helps? Be sure to keep an eye on pending compactions, SSTables count and SSTable per read of course. "alloc" is the memo

Re: Attached profiled data but need help understanding it

Also Attached is a flamed graph generated from a thread dump. On Mon, Feb 27, 2017 at 2:32 AM, Kant Kodali wrote: > Hi, > > Attached are the stats of my Cassandra node running on a 4-core CPU. I am > using sjk-plus tool for the first time so what are the things I should > watched out for in my a