Re: Out of memory issues

2016-05-27 Thread Kai Wang
Paolo, try a few things in cassandra-env.sh 1. HEAP_NEWSIZE="2G". "The 100mb/core commentary in cassandra-env.sh for setting HEAP_NEWSIZE is *wrong*" ( https://tobert.github.io/pages/als-cassandra-21-tuning-guide.html) 2. MaxTenuringThreshold=8 3. enable GC logging (under "# GC logging options --

Re: Out of memory issues

2016-05-27 Thread Paolo Crosato
Hi, thanks for the answer. There were no large insertions and the saved_caches dir had a resonable size. I tried to delete the cashes and set key_cache_size_in_mb to zero, but it didn't help. Today our virtual hardware provided raised cpus to 4, memory to 32GB and doubled the disk size, and th

Re: Out of memory issues

2016-05-26 Thread Mike Yeap
Hi Paolo, a) was there any large insertion done? b) are the a lot of files in the saved_caches directory? c) would you consider to increase the HEAP_NEWSIZE to, say, 1200M? Regards, Mike Yeap On Fri, May 27, 2016 at 12:39 AM, Paolo Crosato < paolo.cros...@targaubiest.com> wrote: > Hi, > > we a

Out of memory issues

2016-05-26 Thread Paolo Crosato
Hi, we are running a cluster of 4 nodes, each one has the same sizing: 2 cores, 16G ram and 1TB of disk space. On every node we are running cassandra 2.0.17, oracle java version "1.7.0_45", centos 6 with this kernel version 2.6.32-431.17.1.el6.x86_64 Two nodes are running just fine, the oth

Re: Out of Memory Issues - SERIOUS

2010-10-07 Thread Jonathan Ellis
if you don't want to lose data, don't wipe your commit logs. that part seems pretty obvious to me. :) cassandra aggressively logs its state when it is running out of memory so you can troubleshoot. look for the GCInspector lines in the log. but in this case it sounds pretty simple; you will be

Out of Memory Issues - SERIOUS

2010-10-07 Thread Dan Hendry
There seems to have been a fair amount of discussion on memory related issues so I apologize if this exact situation has come up before. I am currently in the process of load testing an metrics platform I have written which uses Cassandra and I have run into some very troubling issues. The app