Re: Lot of GC on two nodes out of 7

2016-03-06 Thread Anishek Agarwal
ead of decreasing it should help drop (faster) >> rather than promoting to sv/oldgen (slower) ? >> >> >> >> From: Anishek Agarwal >> Reply-To: "user@cassandra.apache.org" >> Date: Thursday, March 3, 2016 at 8:55 PM >> >> To: "user@cassan

Re: Lot of GC on two nodes out of 7

2016-03-04 Thread Jonathan Haddad
m: Anishek Agarwal > Reply-To: "user@cassandra.apache.org" > Date: Thursday, March 3, 2016 at 8:55 PM > > To: "user@cassandra.apache.org" > Subject: Re: Lot of GC on two nodes out of 7 > > Hello, > > Bryan, most of the partition sizes are under 45 KB > >

Re: Lot of GC on two nodes out of 7

2016-03-03 Thread Jeff Jirsa
at 8:55 PM To: "user@cassandra.apache.org" Subject: Re: Lot of GC on two nodes out of 7 Hello, Bryan, most of the partition sizes are under 45 KB I have tried with concurrent_compactors : 8 for one of the nodes still no improvement, I have tried max_heap_Size : 8G, no improveme

Re: Lot of GC on two nodes out of 7

2016-03-03 Thread Anishek Agarwal
o please check whether you have mutations drop in other nodes or not. >> >> >> >> Hope this helps in your cluster too. >> >> >> >> Regards >> >> Amit Singh >> >> *From:* Jonathan Haddad [mailto:j...@jonhaddad.com] >> *S

Re: Lot of GC on two nodes out of 7

2016-03-03 Thread Bryan Cheng
; > Hope this helps in your cluster too. > > > > Regards > > Amit Singh > > *From:* Jonathan Haddad [mailto:j...@jonhaddad.com] > *Sent:* Wednesday, March 02, 2016 9:33 PM > *To:* user@cassandra.apache.org > *Subject:* Re: Lot of GC on two nodes out of 7 > > &g

RE: Lot of GC on two nodes out of 7

2016-03-02 Thread Amit Singh F
To: user@cassandra.apache.org Subject: Re: Lot of GC on two nodes out of 7 Can you post a gist of the output of jstat -gccause (60 seconds worth)? I think it's cool you're willing to experiment with alternative JVM settings but I've never seen anyone use max tenuring threshold of

RE: Lot of GC on two nodes out of 7

2016-03-02 Thread Amit Singh F
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: Lot of GC on two nodes out of 7 Can you post a gist of the output of jstat -gccause (60 seconds worth)? I think it's cool you're willing to experiment with alternative JVM settings but I've neve

Re: Lot of GC on two nodes out of 7

2016-03-02 Thread Jonathan Haddad
;> >>> 2016-03-02 10:33 GMT+01:00 Anishek Agarwal : >>> >>>> also MAX_HEAP_SIZE=6G and HEAP_NEWSIZE=4G. >>>> >>>> On Wed, Mar 2, 2016 at 1:40 PM, Anishek Agarwal >>>> wrote: >>>> >>>>> Hey Jeff, >>&g

Re: Lot of GC on two nodes out of 7

2016-03-02 Thread Alain RODRIGUEZ
ink thats expected. >>>> >>>> thanks >>>> anishek >>>> >>>> >>>> >>>> >>>> On Wed, Mar 2, 2016 at 1:09 PM, Jeff Jirsa >>>> wrote: >>>> >>>>> Compaction falling beh

Re: Lot of GC on two nodes out of 7

2016-03-02 Thread Anishek Agarwal
s max row size on those nodes >>>> larger than on other nodes? >>>> >>>> What you don’t show in your JVM options is the new gen size – if you do >>>> have unusually large partitions on those two nodes (especially likely if >>>> y

Re: Lot of GC on two nodes out of 7

2016-03-02 Thread Alain RODRIGUEZ
he garbage created by reading large partitions without having to >>> tolerate the promotion. Estimates for the amount of garbage vary, but it >>> could be “gigabytes” of garbage on a very wide partition (see >>> https://issues.apache.org/jira/browse/CASSANDRA-9754 for work in

Re: Lot of GC on two nodes out of 7

2016-03-02 Thread Anishek Agarwal
;> >> - Jeff >> >> From: Anishek Agarwal >> Reply-To: "user@cassandra.apache.org" >> Date: Tuesday, March 1, 2016 at 11:12 PM >> To: "user@cassandra.apache.org" >> Subject: Lot of GC on two nodes out of 7 >> >> Hello, >&

Re: Lot of GC on two nodes out of 7

2016-03-02 Thread Anishek Agarwal
progress > to help mitigate that type of pain). > > - Jeff > > From: Anishek Agarwal > Reply-To: "user@cassandra.apache.org" > Date: Tuesday, March 1, 2016 at 11:12 PM > To: "user@cassandra.apache.org" > Subject: Lot of GC on two nodes out of 7 > > Hello,

Re: Lot of GC on two nodes out of 7

2016-03-01 Thread Jeff Jirsa
mitigate that type of pain). - Jeff From: Anishek Agarwal Reply-To: "user@cassandra.apache.org" Date: Tuesday, March 1, 2016 at 11:12 PM To: "user@cassandra.apache.org" Subject: Lot of GC on two nodes out of 7 Hello, we have a cassandra cluster of 7 nodes, all of them

Lot of GC on two nodes out of 7

2016-03-01 Thread Anishek Agarwal
Hello, we have a cassandra cluster of 7 nodes, all of them have the same JVM GC configurations, all our writes / reads use the TokenAware Policy wrapping a DCAware policy. All nodes are part of same Datacenter. We are seeing that two nodes are having high GC collection times. Then mostly seem to