Reid, Jon, thanks for the feedback and comments. Interesting readings.
https://jira.apache.org/jira/browse/CASSANDRA-9766 is basically describing
exactly the same what we are experiencing, namely e.g. unthrottling not
changing anything at all, thus I simply take it as Cassandra itself is limitin
Thanks for the reading Jon. 😊
From: Jon Haddad
Reply-To: "user@cassandra.apache.org"
Date: Tuesday, October 22, 2019 at 12:32 PM
To: "user@cassandra.apache.org"
Subject: Re: Cassandra 2.1.18 - Question on stream/bootstrap throughput
Message from External Sender
CPU waiting on memory will look
CPU waiting on memory will look like CPU overhead. There's a good post on
the topic by Brendan Gregg:
http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-is-wrong.html
Regarding GC, I agree with Reid. You're probably not going to saturate
your network card no matter what your settings,
Thomas, what is your frequency of metric collection? If it is minute-level
granularity, that can give a very false impression. I’ve seen CPU and disk
throttles that don’t even begin to show visibility until second-level
granularity around the time of the constraining event. Even clearer is 10
Hi Alex,
Increased streaming throughput has been set on the existing nodes only, cause
it is meant to limit outgoing traffic only, right? At least when judging from
the name, reading the documentation etc.
Increased compaction throughput on all nodes, although my understanding is that
it would
On Tue, Oct 22, 2019 at 12:47 PM Steinmaurer, Thomas <
thomas.steinmau...@dynatrace.com> wrote:
>
>
> using 2.1.8, 3 nodes (m4.10xlarge, ESB SSD-based), vnodes=256, RF=3, we
> are trying to add a 4th node.
>
>
>
> The two options to my knowledge, mainly affecting throughput, namely
> stream output
A high level of compaction seems highly likely to throttle you by sending the
service into a GC death spiral, doubly-so if any repairs happen to be underway
at the same time (I may or may not have killed a few nodes this way, but I
admit nothing!). Even if not in GC hell, it can cause you to ep
Hello,
using 2.1.8, 3 nodes (m4.10xlarge, ESB SSD-based), vnodes=256, RF=3, we are
trying to add a 4th node.
The two options to my knowledge, mainly affecting throughput, namely stream
output and compaction throttling has been set to very high values (e.g. stream
output = 800 Mbit/s resp. comp
Did you check nodetool status and logs? If so, what is reported?
Regarding that more and more memory is used. This might be a problem with your
table design. I would start analyzing nodetool tablestats output. It reports
how much memory (especially off heap) is used by which table.
Best,
Matthi
