Re: At what point does the cluster get faster than the individual nodes?

2010-04-21 Thread Mike Gallamore
Some people might be able to answer this better than me. However: with quorum consistency you have to communicate with n/2 + 1 where n is the replication factor nodes. So unless you are disk bound your real expense is going to be all those extra network latencies. I'd expect that you'll see a r

Re: Regarding Cassandra Scalability

2010-04-16 Thread Mike Gallamore
02:22 PM, Stu Hood wrote: http://twitter.com/jromeh/status/12295736793 -Original Message- From: "Mike Gallamore" Sent: Friday, April 16, 2010 3:46pm To: user@cassandra.apache.org Subject: Re: Regarding Cassandra Scalability Also people with 1M followers tend to have "public&

Re: Regarding Cassandra Scalability

2010-04-16 Thread Mike Gallamore
Also people with 1M followers tend to have "public" tweets, which means really I think it would be the same as subscribing to a RSS feed or whatever. You aren't getting a local copy because you will "always" have access to the tweet as will everyone else. Also tweets don't change AFAIK so no po

Re: effective modeling for fixed limit columns

2010-04-16 Thread Mike Gallamore
The problem I'm working on is very similar to this. I'm working on a reputation system and we keep a fixed number of day buckets for the scores. So when new data comes in you need to find out what bucket is supposed to be used, remove the data in it if you've moved to a new bucket as the data t

Re: Regarding Cassandra Scalability

2010-04-16 Thread Mike Gallamore
On 04/16/2010 01:38 AM, dir dir wrote: I hear Facebook.com and tweeter.com using cassandra database. In my opinion Facebook and tweeter have hundreds TB data. because their user reach hundreds million people. I think you might be forgetting just how tiny tweets are. The las

Re: History values

2010-04-14 Thread Mike Gallamore
Here here on documentation. For example thrift examples in python and java. That is great but I've never coded in either (and am limited to perl or C at work because when have 5 years worth of code and experience with other modules provided for those languages). So I'm stuck with whatever the

Re: How to perform queries on Cassandra?

2010-04-09 Thread Mike Gallamore
I apologize in advance if this goes into esoteric algorithms a bit too much but I think this will get to an interesting idea to solve your problem. My background is physics particularly computer simulations of complex systems. Anyways in cosmology an interesting algorithm is called an n-body tr

Re: writes to Cassandra failing occasionally

2010-04-09 Thread Mike Gallamore
" was that feedbacks into the system for updates of the value. Anyways lots of parallelism opportunities. 2010/4/9 Ted Zlatanov > On Thu, 08 Apr 2010 11:50:38 -0700 Mike Gallamore < > mike.e.gallam...@googlemail.com> wrote: > > MG> Yes I agree single threaded is proba

Re: writes to Cassandra failing occasionally

2010-04-08 Thread Mike Gallamore
On 04/08/2010 11:46 AM, Jeremy Dunck wrote: On Thu, Apr 8, 2010 at 12:41 PM, Mike Gallamore wrote: Hello. If you are doing exactly the same thing as N::C::Easy (ie a join on the gettimeofday). Then you should have the same problem I found a fix for. The problem is that the microseconds

Re: writes to Cassandra failing occasionally

2010-04-08 Thread Mike Gallamore
On 04/08/2010 05:53 AM, Ted Zlatanov wrote: On Thu, 08 Apr 2010 12:53:48 +0100 Philip Jackson wrote: PJ> At Wed, 07 Apr 2010 13:19:26 -0700, PJ> Mike Gallamore wrote: I have writes to cassandra that are failing, or at least a read shortly after a write is still getting an old va

Re: writes to Cassandra failing occasionally

2010-04-08 Thread Mike Gallamore
hough? I guess I still would be blocking on ram writes but isn't like there is multiple CPUs I need to keep busy or anything. On 04/08/2010 06:07 AM, Ted Zlatanov wrote: On Wed, 07 Apr 2010 13:19:26 -0700 Mike Gallamore wrote: MG> As an aside I motified some other code to use Net::Cassan

Re: writes to Cassandra failing occasionally

2010-04-08 Thread Mike Gallamore
On 04/08/2010 04:53 AM, Philip Jackson wrote: At Wed, 07 Apr 2010 13:19:26 -0700, Mike Gallamore wrote: I have writes to cassandra that are failing, or at least a read shortly after a write is still getting an old value. I realize Cassandra is "eventually consistent" but this s

Re: writes to Cassandra failing occasionally

2010-04-08 Thread Mike Gallamore
On 04/07/2010 01:31 PM, Eric Evans wrote: On Wed, 2010-04-07 at 13:19 -0700, Mike Gallamore wrote: I have writes to cassandra that are failing, or at least a read shortly after a write is still getting an old value. I realize Cassandra is "eventually consistent" but this system i

Re: writes to Cassandra failing occasionally

2010-04-08 Thread Mike Gallamore
On 04/08/2010 04:53 AM, Philip Jackson wrote: At Wed, 07 Apr 2010 13:19:26 -0700, Mike Gallamore wrote: I have writes to cassandra that are failing, or at least a read shortly after a write is still getting an old value. I realize Cassandra is "eventually consistent" but this s

writes to Cassandra failing occasionally

2010-04-07 Thread Mike Gallamore
I have writes to cassandra that are failing, or at least a read shortly after a write is still getting an old value. I realize Cassandra is "eventually consistent" but this system is a single CPU single node with consistency level set to 1, so this seems odd to me. My setup: Cassandra 0.6rc1

Re: Net::Cassandra::Easy deletion failed

2010-04-06 Thread Mike Gallamore
On 04/06/2010 01:36 PM, Ted Zlatanov wrote: On Tue, 06 Apr 2010 13:24:45 -0700 Mike Gallamore wrote: MG> Thanks for the reply. The newest version of the module I see on CPAN MG> is 0.08b. I actually had 0.07 installed and am using 0.6beta3 for MG> cassandra. Is there somewhe

Re: Net::Cassandra::Easy deletion failed

2010-04-06 Thread Mike Gallamore
ee if that helps. On 04/06/2010 01:14 PM, Ted Zlatanov wrote: On Tue, 06 Apr 2010 11:07:03 -0700 Mike Gallamore wrote: MG> Seems to be internal to java/cassandra itself. MG> I have some tests and I want to make sure that I have a "clean slate" MG> each time I run the test.

problem with Net::Cassanda::Easy deleting columns

2010-04-06 Thread Mike Gallamore
Hello I tried to post this earlier but something seems to have gone wrong with sending the message. I have a test perl script that I'm using to test the behaviour of some of my existing code. It is important that the values start in a clean state at the beginning of the tests, as I'm incrementing

Net::Cassandra::Easy deletion failed

2010-04-06 Thread Mike Gallamore
Seems to be internal to java/cassandra itself. I have some tests and I want to make sure that I have a "clean slate" each time I run the test. Clean as far as my code cares is that "value" is not defined. I'm running "bin/cassandra -f" with the default install/options. So at the beginning of

Re: Deployment on AWS and replication strategies

2010-04-04 Thread Mike Gallamore
aid I'll take a look and see if I can figure out how it works. I have coded in C/C++ so I probably can handle the logic part of Java code okay. On 2010-04-04, at 1:18 AM, Benjamin Black wrote: > On Sat, Apr 3, 2010 at 8:23 PM, Mike Gallamore > wrote: >>> >> I didn

Re: Deployment on AWS and replication strategies

2010-04-03 Thread Mike Gallamore
Hi Benjamin, Thanks for the reply. On 2010-04-03, at 8:12 PM, Benjamin Black wrote: > On Sat, Apr 3, 2010 at 3:41 PM, Mike Gallamore > wrote: >> >> Useful things that nodes could advertise: >> >> data-centre they are in, > > This is what the snitches d

Re: Deployment on AWS and replication strategies

2010-04-03 Thread Mike Gallamore
Hi everyone, At my work we are in the early stages of moving our data which lives on EC2 machines from a Flare/memcache system to Cassandra so your chat has been interesting to me. I realize that this might complicate things and make things less "simple" but would it be useful for the nodes th

Re: expiring data out of Cassandra/time to live

2010-03-31 Thread Mike Gallamore
if Java scares you. :) On Wed, Mar 31, 2010 at 1:39 PM, Mike Gallamore wrote: Hello everyone, I saw a thread on the incubator user chat that started a few months ago: http://www.mail-archive.com/cassandra-u...@incubator.apache.org/msg02047.html . It looks like this is the new official user

expiring data out of Cassandra/time to live

2010-03-31 Thread Mike Gallamore
Hello everyone, I saw a thread on the incubator user chat that started a few months ago: http://www.mail-archive.com/cassandra-u...@incubator.apache.org/msg02047.html . It looks like this is the new official user mailing list so I'll add my thoughts/question here. Is there any way to set a T