>
> > We have a 100 nodes ish cluster, I find that there are dropped messages
> on
> > random nodes in the cluster, which caused error spikes and P99 latency
> > spikes as well.
> >
> > I tried to figure out the cause. I do not see any obvious bottleneck in
>
on-gc pause is also troubling. (I assume non-gc since there
wasn't anything logged by the GC inspector.)
On Mon, Jan 23, 2017 at 12:36 AM, Dikang Gu wrote:
> Hello there,
>
> We have a 100 nodes ish cluster, I find that there are dropped messages on
> random nodes in the cluster, whi
*
Engineering Manager CDE
*(408) 438-3156 - mobile*
On Mon, Jan 23, 2017 at 4:55 PM, Blake Eggleston
wrote:
> Hi Dikang,
>
> Do you have any GC logging or metrics you can correlate with the dropped
> messages? A 13 second pause sounds like a bad GC pause.
>
> Thanks,
Hi Dikang,
Do you have any GC logging or metrics you can correlate with the dropped
messages? A 13 second pause sounds like a bad GC pause.
Thanks,
Blake
On January 22, 2017 at 10:37:22 PM, Dikang Gu (dikan...@gmail.com) wrote:
Btw, the C* version is 2.2.5, with several backported patches
Btw, the C* version is 2.2.5, with several backported patches.
On Sun, Jan 22, 2017 at 10:36 PM, Dikang Gu wrote:
> Hello there,
>
> We have a 100 nodes ish cluster, I find that there are dropped messages on
> random nodes in the cluster, which caused error spikes and P99 latency
Hello there,
We have a 100 nodes ish cluster, I find that there are dropped messages on
random nodes in the cluster, which caused error spikes and P99 latency
spikes as well.
I tried to figure out the cause. I do not see any obvious bottleneck in the
cluster, the C* nodes still have plenty of
ible to tell from logs what the
>>> message types (verb) were dropped. I read this was changed for spamming,
>>> but I think the behavior should be configurable, either aggregate counts
>>> of
>>> dropped messages or log individual occurrences with the message verb.
2010 at 8:29 AM, Carl Bruecken
>> wrote:
>>>
>>> With current implementation, it's impossible to tell from logs what the
>>> message types (verb) were dropped. I read this was changed for spamming,
>>> but I think the behavior should be configurable, e
as changed for spamming,
but I think the behavior should be configurable, either aggregate counts of
dropped messages or log individual occurrences with the message verb.
One suggestion is to pass message into
MessagingService.incrementDroppedMessages and have a configuration item or
system propert
think the behavior should be configurable, either aggregate counts of
> dropped messages or log individual occurrences with the message verb.
>
> One suggestion is to pass message into
> MessagingService.incrementDroppedMessages and have a configuration item or
> system property indica
With current implementation, it's impossible to tell from logs what
the message types (verb) were dropped. I read this was changed for
spamming, but I think the behavior should be configurable, either
aggregate counts of dropped messages or log individual occurrences with
the message
11 matches
Mail list logo
