Re: UnavailableException when data grows

2010-10-02 Thread Peter Schuller
> And I'm still getting UnavailableException and TimedOutException when there > Cassandra daemon is doing either Compaction or Garbage collection... Have you specifically correlated this? If so, which one, or both? GC should not cause unavailable exceptions on a healthy cluster with healthy nodes

Re: UnavailableException when data grows

2010-10-01 Thread Rana Aich
And I'm still getting UnavailableException and TimedOutException when there Cassandra daemon is doing either Compaction or Garbage collection... On Thu, Sep 30, 2010 at 2:42 PM, Rana Aich wrote: > I ran the nodetool cleanup...but the scenario doesn't change... > > > On Thu, Sep 30, 2010 at 1:14

Re: UnavailableException when data grows

2010-09-30 Thread Rana Aich
I ran the nodetool cleanup...but the scenario doesn't change... On Thu, Sep 30, 2010 at 1:14 PM, Edward Capriolo wrote: > After nodetool move you have to run nodetool cleanup. > > On Thu, Sep 30, 2010 at 3:45 PM, Rana Aich wrote: > > I have arranged my initial tokens and get this result: > > Add

Re: UnavailableException when data grows

2010-09-30 Thread Edward Capriolo
After nodetool move you have to run nodetool cleanup. On Thu, Sep 30, 2010 at 3:45 PM, Rana Aich wrote: > I have arranged my initial tokens and get this result: > Address       Status     Load          Range >      Ring > > 17014118346046923173168730371588000 > 192.168.202.1 Up         208.39

Re: UnavailableException when data grows

2010-09-30 Thread Rana Aich
I have arranged my initial tokens and get this result: Address Status Load Range Ring 17014118346046923173168730371588000 192.168.202.1 Up 208.39 GB 3402823669209384634633746074317700 |<--| 192.168.202.2 Up 333.52 GB 6805647338418769269267492148

Re: UnavailableException when data grows

2010-09-27 Thread Oleg Anastasyev
Rana Aich gmail.com> writes: > > Yet my nodetool shows the following: > > 192.168.202.202Down       319.94 GB     7200044730783885730400843868815072654      |<--| > 192.168.202.4 Up         382.39 GB     23719654286404067863958492664769598669     |   ^ > 192.168.202.2 Up         106.81 GB     3

Re: UnavailableException when data grows

2010-09-27 Thread Benjamin Black
Your ring is wildly unbalanced and you are almost certainly out of I/O on one or more nodes. You should be monitoring via JMX and common systems tools to know when you are starting to have issues. It is going to take you some effort to get out of this situation now. b On Mon, Sep 27, 2010 at 2

Re: UnavailableException when data grows

2010-09-27 Thread Rana Aich
Hi Peter, Thanks for your detailed query... I have 8 m/c cluster. KVSHIGH1,2,3,4 and KVSLOW1,2,3,4. As the name suggests KVSLOWs have low diskspace ~ 350GB Whereas KVSHIGHs have 1.5 terabytes. Yet my nodetool shows the following: 192.168.202.202Down 319.94 GB 72000447307838857304008438688

Re: UnavailableException when data grows

2010-09-27 Thread Peter Schuller
> How can I handle this kind of situation? In terms of surviving the problem, a re-try on the client side might help assuming the problem is temporary. However, certainly the fact that you're seeing an issue to begin with is interesting, and the way to avoid it would depend on what the problem i

UnavailableException when data grows

2010-09-27 Thread Rana Aich
Hi, I'm having great difficulty in inserting data in 8 server Cassandra cluster (RandomPartition with RF 2). For the first one billion the data insertion was smooth. But slowly I'm getting Unavailable Exception from the cluster. And now I can't put not more than 30 million data at a one stretch be