Cassandra 0.5.1 slow down after doing a lot of inserts

2010-04-16 Thread Lu Ming 
Hi:
   We have build a  storage system with 4 nodes in the cluster. We use the 
default configuration file,
every node have 2*E5504 CPU, 8G memory and 6*1T Sata disk.  The cluster stores 
about 600G data.

   We found that Cassandra 0.5.1 ALWAYS slow down after a lot of inserts. When 
it happens, CPU and Disk load
are very low. We use "tpstats" nodetools and find that more than 300,000  
row-mutations is still pending and
the write speed is no more than 100~200 row-mutations per second.
  It also happens when  Cassandra starts and replays a large commitlog file. So 
we must cost several hours to
wait it finishs startup.

  The Faq of Cassandra tells that it is caused by GC?  any other explaining?


thanks a lot.

Re: Cassandra 0.5.1 slow down after doing a lot of inserts

2010-04-16 Thread Lu Ming 
Cassandra often take >30 minutes to write about 300,000 row-mutations when 
startup and replaying commitlog!


--
From: "Lu Ming" 
Sent: Friday, April 16, 2010 11:33 PM
To: 
Subject:  Cassandra 0.5.1 slow down after doing a lot of inserts


Hi:
  We have build a  storage system with 4 nodes in the cluster. We use the 
default configuration file,
every node have 2*E5504 CPU, 8G memory and 6*1T Sata disk.  The cluster 
stores about 600G data.


  We found that Cassandra 0.5.1 ALWAYS slow down after a lot of inserts. 
When it happens, CPU and Disk load
are very low. We use "tpstats" nodetools and find that more than 300,000 
row-mutations is still pending and

the write speed is no more than 100~200 row-mutations per second.
 It also happens when  Cassandra starts and replays a large commitlog 
file. So we must cost several hours to

wait it finishs startup.

 The Faq of Cassandra tells that it is caused by GC?  any other 
explaining?



thanks a lot.