Re: Bootstrap OOM issues with Cassandra 3.11.1

2018-08-07 Thread Laszlo Szabo
hey're holding a ton offheap (either in > bloom filter, memtable, etc). With what JVM args are you starting > cassandra (how much direct memory are you allocating)? Are all of your OOMs > in direct memory? > > > > On Tue, Aug 7, 2018 at 6:30 AM, Laszlo Szabo < > laszlo

Re: Bootstrap OOM issues with Cassandra 3.11.1

2018-08-07 Thread Laszlo Szabo
Aug 6, 2018 at 8:57 PM, Jeff Jirsa wrote: > > > Upgrading to 3.11.3 May fix it (there were some memory recycling bugs > fixed recently), but analyzing the heap will be the best option > > If you can print out the heap histogram and stack trace or open a heap > dump in your ki

Bootstrap OOM issues with Cassandra 3.11.1

2018-08-06 Thread Laszlo Szabo
Hello All, I'm having JVM unstable / OOM errors when attempting to auto bootstrap a 9th node to an existing 8 node cluster (256 tokens). Each machine has 24 cores 148GB RAM and 10TB (2TB used). Under normal operation the 8 nodes have JVM memory configured with Xms35G and Xmx35G, and handle 2-4