Re: Cassandra nodes failing with OOM

2012-11-26 Thread Ивaн Cобoлeв
please contact the sender immediately and > irrevocably delete this message and any copies. > > *From:* some.unique.lo...@gmail.com [mailto:some.unique.lo...@gmail.com] > *On Behalf Of *Ивaн Cобoлeв > *Sent:* Saturday, November 17, 2012 08:08 > *To:* user@cassandra.apache.org > *Subject

RE: Cassandra nodes failing with OOM

2012-11-19 Thread Viktor Jevdokimov
turday, November 17, 2012 08:08 To: user@cassandra.apache.org Subject: Cassandra nodes failing with OOM Dear Community, advice from you needed. We have a cluster, 1/6 nodes of which died for various reasons(3 had OOM message). Nodes died in groups of 3, 1, 2. No adjacent died, though we use Sim

Re: Cassandra nodes failing with OOM

2012-11-19 Thread Janne Jalkanen
Something that bit us recently was the size of bloom filters: we have a column family which is mostly written to, and only read sequentially, so we were able to free a lot of memory and decrease GC pressure by increasing bloom_filter_fp_chance for that particular CF. This on 1.0.12. /Janne O

Re: Cassandra nodes failing with OOM

2012-11-18 Thread aaron morton
> 1. How much GCInspector warnings per hour are considered 'normal'? None. A couple during compaction or repair is not the end of the world. But if you have enough to thinking about "per hour" it's too many. > 2. What should be the next thing to check? Try to determine if the GC activity correl

Cassandra nodes failing with OOM

2012-11-16 Thread Ивaн Cобoлeв
Dear Community, advice from you needed. We have a cluster, 1/6 nodes of which died for various reasons(3 had OOM message). Nodes died in groups of 3, 1, 2. No adjacent died, though we use SimpleSnitch. Version: 1.1.6 Hardware: 12Gb RAM / 8 cores(virtual) Data: 40Gb/node