The last run I attempted used 135GB of RAM allocated to the JVM (arguments
below), and while there are OOM errors, there is not a stack trace in
either the system or debug log. On direct memory runs, there is a stack
trace. The last Direct memory run used 60GB heaps and 60GB for off heap
(that wa
That's a direct memory OOM - it's not the heap, it's the offheap.
You can see
that gpsmessages.addressreceivedtime_idxgpsmessages.addressreceivedtime_idx
is holding about 2GB of offheap memory (most of it for the bloom filter),
but none of the others look like they're holding a ton offheap (either
By default Cassandra is set to generate a heap dump on OOM. It can be a bit
tricky to figure out what’s going on exactly but it’s the best evidence you
can work with.
On Tue, Aug 7, 2018 at 6:30 AM Laszlo Szabo
wrote:
> Hi,
>
> Thanks for the fast response!
>
> We are not using any materialized
Hi,
Thanks for the fast response!
We are not using any materialized views, but there are several indexes. I
don't have a recent heap dump, and it will be about 24 before I can
generate an interesting one, but most of the memory was allocated to byte
buffers, so not entirely helpful.
nodetool cf
Upgrading to 3.11.3 May fix it (there were some memory recycling bugs fixed
recently), but analyzing the heap will be the best option
If you can print out the heap histogram and stack trace or open a heap dump in
your kit or visualvm or MAT and show us what’s at the top of the reclaimed
objec
Are you using materialized views or secondary indices?
--
Jeff Jirsa
> On Aug 6, 2018, at 3:49 PM, Laszlo Szabo
> wrote:
>
> Hello All,
>
> I'm having JVM unstable / OOM errors when attempting to auto bootstrap a 9th
> node to an existing 8 node cluster (256 tokens). Each machine has 24
Hello All,
I'm having JVM unstable / OOM errors when attempting to auto bootstrap a
9th node to an existing 8 node cluster (256 tokens). Each machine has 24
cores 148GB RAM and 10TB (2TB used). Under normal operation the 8 nodes
have JVM memory configured with Xms35G and Xmx35G, and handle 2-4
