Re: question about replicas & dynamic response to load

2011-03-05 Thread aaron morton
Agree. Cassandra generally assumes a reasonable static cluster membership. There are some tricks that can be done with copying SSTables but they will only reduce the need to stream data around, not eliminate it. This may not suit your problem domain but, speaking of the AWS infrastructure how a

Re: Error when bringing up nodes during failure testing

2011-03-05 Thread mcasandra
aaron morton wrote: > > Can you include the full error stack ? > > It's failing because of the reason stated. But I need some more info to > understand what part of the startup process it's stuck at. > > Thanks for responding! I'll send it as soon as I can get on my network. But you mentione

Re: Error when bringing up nodes during failure testing

2011-03-05 Thread aaron morton
Can you include the full error stack ? It's failing because of the reason stated. But I need some more info to understand what part of the startup process it's stuck at. Aaron On 4/03/2011, at 6:39 AM, mcasandra wrote: > Whenever I do failure testing I see this error message and then cassa

Re: Reducing memory footprint

2011-03-05 Thread aaron morton
There is some additional memory usage in the JVM beyond that Heap size, in the permanent generation. 900mb sounds like too much for that, but you can check by connecting with JConsole and looking at the memory tab. You can also check the heap size there to see that it's under the value you've se

Re: Nodes frozen in GC

2011-03-05 Thread aaron morton
Your node is under memory pressure, after the GC there is still 5.7GB in use. In fact it looks like memory usage went up during the GC process. Can you reduce the memtable size, caches or the number of CF's or increase the JVM size? Also is this happening under heavy load ? Aaron On 6/03/201

Re: cassandra in-production experiences with .7 series

2011-03-05 Thread aaron morton
Hi paul, 0.7 has had some problems but I think it's settling down. Hopefully some more people with public facing sites can comment. WRT some of your other questions: - I'm not aware of any specific performance issues with multi gets (multiget_slice) can you be more specific. - You can implemen

Re: OOM exceptions

2011-03-05 Thread aaron morton
First question is which version are you running ? Am guessing 0.6 something If you have OOM in the compaction thread it may be because of a very large row. The CF information available through JConsole will give you the max row size for the CF. Your setting for RowWarningThresholdInMB is 512, h

cassandra in-production experiences with .7 series

2011-03-05 Thread Paul Pak
Hello all, I was wondering if people could share their overall experiences with using .7 series of Cassandra in production? Is anyone using it? I see a lot of people have troubles on the list, but it's hard to see which issues get resolved and which remain. So I was wondering how many people ac

Nodes frozen in GC

2011-03-05 Thread ruslan usifov
Hello I have cluster of 3 cassandra 0.7.3 nodes, and some times nodes a frozen. As i understand this because GC, in this moment i see INFO [ScheduledTasks:1] 2011-03-05 15:21:23,524 GCInspector.java (line 128) GC for ConcurrentMarkSweep: 18052 ms, -997761672 reclaimed leaving 5796586088 How can

Re: Argh: Data Corruption (LOST DATA) (0.7.0)

2011-03-05 Thread Terje Marthinussen
Hi, Unfortunately, this patch is already included in the build I have. Thanks for the suggestion though! Terje On Sat, Mar 5, 2011 at 7:47 PM, Sylvain Lebresne wrote: > Also, if you can, please be sure to try the new 0.7.3 release. We had a bug > with the compaction of superColumns for instance

Re: [RELEASE] 0.7.3

2011-03-05 Thread Sylvain Lebresne
If I may add, people that wish to upgrade from 0.6 to 0.7 are also strongly encouraged to go directly to 0.7.3, otherwise they would be exposed to the bug that 0.7.3 fixes. -- Sylvain On Fri, Mar 4, 2011 at 10:52 PM, Eric Evans wrote: > > It's only been a couple of weeks since the last release,

Re: Argh: Data Corruption (LOST DATA) (0.7.0)

2011-03-05 Thread Sylvain Lebresne
Also, if you can, please be sure to try the new 0.7.3 release. We had a bug with the compaction of superColumns for instance that is fixed there ( https://issues.apache.org/jira/browse/CASSANDRA-2104). It also ships with a new scrub command that tries to find if your sstables are corrupted and repa