Re: Cassandra node JVM hang during node repair a table with materialized view

2020-04-17 Thread Reid Pinchback
l 16, 2020 at 3:32 AM To: "user@cassandra.apache.org" Subject: Re: Cassandra node JVM hang during node repair a table with materialized view Message from External Sender Thanks a lot. We are working on removing views and control the partition size. I hope the improvements help us Bes

Re: Cassandra node JVM hang during node repair a table with materialized view

2020-04-16 Thread Ben G
Thanks a lot. We are working on removing views and control the partition size. I hope the improvements help us Best regards Gb Erick Ramirez 于2020年4月16日周四 下午2:08写道: > GC collector is G1. I ever repair the node after scale up. The JVM issue >> reproduced. Can I increase the heap to 40 GB on

Re: Cassandra node JVM hang during node repair a table with materialized view

2020-04-15 Thread Erick Ramirez
> > GC collector is G1. I ever repair the node after scale up. The JVM issue > reproduced. Can I increase the heap to 40 GB on a 64GB VM? > I wouldn't recommend going beyond 31GB on G1. It will be diminishing returns as I mentioned before. Do you think the issue is related to materialized view

Re: Cassandra node JVM hang during node repair a table with materialized view

2020-04-15 Thread Ben G
Thanks a lot for your sharing. The node is added recently. The bootstrap failed since too many tombstone. So we enabled the node without bootstrap enabled. Some sstables are not created in bootstrap. So the missing files might be numerous. I have set the repair thread number is 1. should I als

Re: Cassandra node JVM hang during node repair a table with materialized view

2020-04-15 Thread Erick Ramirez
Is this the first time you've repaired your cluster? Because it sounds like it isn't coping. First thing you need to make sure of is to *not* run repairs in parallel. It can overload your cluster -- only kick off a repair one node at a time on small clusters. For larger clusters, you might be able

Cassandra node JVM hang during node repair a table with materialized view

2020-04-15 Thread Ben G
Hello experts I have a 9 nodes cluster on AWS. Recently, some nodes were down and I want to repair the cluster after I restarted them. But I found the repair operation causes lots of memtable flush and then the JVM GC failed. Consequently, the node hang. I am using the cassandra 3.1.0. java vers