Re: with proof Re: cassandra goes infinite loop and data lost.....

2011-07-21 Thread Jonathan Ellis
You should be able to tell from earlier in the log if this is from a request, from hinted handoff replay, or something else On Wed, Jul 20, 2011 at 10:42 PM, Yan Chunlu wrote: > thans for the reply. > now the problem is how can I get rid of the ""N of 2147483647 ", it seems > never ends, and the

Re: with proof Re: cassandra goes infinite loop and data lost.....

2011-07-20 Thread Yan Chunlu
thans for the reply. now the problem is how can I get rid of the ""N of 2147483647 ", it seems never ends, and the node never goes UP last time it happens I run "node cleanup", turns out some data loss(not sure if caused by cleanup). On Thu, Jul 21, 2011 at 11:37 AM, aaron morton wrote: > Pe

Re: with proof Re: cassandra goes infinite loop and data lost.....

2011-07-20 Thread aaron morton
Personally I would do a repair first if you need to do one, just so you are confident everything is where is should be. Then do the move as described in the wiki. Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 21 Jul 2011, at

Re: with proof Re: cassandra goes infinite loop and data lost.....

2011-07-20 Thread Yan Chunlu
sorry for the misunderstanding. I saw many N of 2147483647 which N=0 and thought it was not doing anything. my node was very unbalanced and I was intend to rebalance it by "nodetool move" after a "node repair", does that cause the slices much large? Address Status State Load

Re: with proof Re: cassandra goes infinite loop and data lost.....

2011-07-20 Thread Jonathan Ellis
This is not an infinite loop, you can see the column objects being iterated over are different. Like I said last time, "I do see that it's saying "N of 2147483647" which looks like you're doing slices with a much larger limit than is advisable." On Wed, Jul 20, 2011 at 9:00 PM, Yan Chunlu wrote:

with proof Re: cassandra goes infinite loop and data lost.....

2011-07-20 Thread Yan Chunlu
this time it is another node, the node goes down during repair, and come back but never up, I change log level to "DEBUG" and found out it print out the following message infinitely DEBUG [main] 2011-07-20 20:58:16,286 SliceQueryFilter.java (line 123) collecting 0 of 2147483647: 76616c7565:false:6

Re: cassandra goes infinite loop and data lost.....

2011-07-13 Thread Yan Chunlu
okay, I am not sure if it is infinite loop, I change log4j to "DEBUG" only because cassandra never get online after run cassandra, it seems just halt. I enable debug then it start showing those message very fast and never end. I have just run nodetool cleanup, and it start reading commitlog, seem

Re: cassandra goes infinite loop and data lost.....

2011-07-13 Thread Jonathan Ellis
That says "I'm collecting data to answer requests." I don't see anything here that indicates an infinite loop. I do see that it's saying "N of 2147483647" which looks like you're doing slices with a much larger limit than is advisable (good way to OOM the way you already did). On Wed, Jul 13, 20

Re: cassandra goes infinite loop and data lost.....

2011-07-13 Thread Yan Chunlu
problem is I can't take cassandra back does that because not enough memory for cassandra? On Thu, Jul 14, 2011 at 11:29 AM, Bret Palsson wrote: > How much total memory does your machine have? > > -- > Bret > > On Wednesday, July 13, 2011 at 9:27 PM, Yan Chunlu wrote: > > I gave cassandra 8GB

Re: cassandra goes infinite loop and data lost.....

2011-07-13 Thread Yan Chunlu
16GB On Thu, Jul 14, 2011 at 11:29 AM, Bret Palsson wrote: > How much total memory does your machine have? > > -- > Bret > > On Wednesday, July 13, 2011 at 9:27 PM, Yan Chunlu wrote: > > I gave cassandra 8GB heap size and somehow it run out of memory and > crashed. after I start it, it just run

Re: cassandra goes infinite loop and data lost.....

2011-07-13 Thread Bret Palsson
How much total memory does your machine have? -- Bret On Wednesday, July 13, 2011 at 9:27 PM, Yan Chunlu wrote: > I gave cassandra 8GB heap size and somehow it run out of memory and crashed. > after I start it, it just runs in to the following infinite loop, the last > line: > DEBUG [main] 2

Re: cassandra goes infinite loop and data lost.....

2011-07-13 Thread Yan Chunlu
I gave cassandra 8GB heap size and somehow it run out of memory and crashed. after I start it, it just runs in to the following infinite loop, the last line: DEBUG [main] 2011-07-13 22:19:00,586 SliceQueryFilter.java (line 123) collecting 0 of 2147483647: 100zs:false:14@1310168625866434 goes for e

cassandra goes infinite loop and data lost.....

2011-07-13 Thread Yan Chunlu
DEBUG [main] 2011-07-13 22:19:00,586 SliceQueryFilter.java (line 123) collecting 0 of 2147483647: 100zs:false:14@1310168625866434