Re: Using C* and CAS to coordinate workers

2014-04-04 Thread Jan Algermissen
Hi Duy Hai, On 04 Apr 2014, at 20:48, DuyHai Doan wrote: > @Jan > > Your use-case is different than what i though. So basically you have only > one data source (the feed) and many consumers (the workers) > > Only one worker is allowed to consumer the feed at a time. > > This can be model

Re: Using C* and CAS to coordinate workers

2014-04-04 Thread DuyHai Doan
@Jan Your use-case is different than what i though. So basically you have only one data source (the feed) and many consumers (the workers) Only one worker is allowed to consumer the feed at a time. This can be modeled very easily using distributed lock with C.A.S *CREATE TABLE feed_lock (*

Re: Using C* and CAS to coordinate workers

2014-04-04 Thread Jan Algermissen
Hi DuyHai, On 04 Apr 2014, at 13:58, DuyHai Doan wrote: > @Jan > > This subject of distributed workers & queues has been discussed in the > mailing list many times. Sorry + thanks. Unfortunately, I do not want to use C* as a queue, but to coordinate workers that page through an (XML) data

Re: Using C* and CAS to coordinate workers

2014-04-04 Thread DuyHai Doan
@Jan This subject of distributed workers & queues has been discussed in the mailing list many times. Basically one implementation can be: 1) *p* data providers, *c* data consumers 2) create partitions (physical rows) of arbitrary number of columns (let's say 10 000, not too big though). Partitio

Re: Using C* and CAS to coordinate workers

2014-04-04 Thread prem yadav
Oh ok. I thought you did not have a cassandra cluster already. Sorry about that. On Fri, Apr 4, 2014 at 11:42 AM, Jan Algermissen wrote: > > On 04 Apr 2014, at 11:18, prem yadav wrote: > > Though cassandra can work but to me it looks like you could use a > persistent queue for example (rabbitM

Re: Using C* and CAS to coordinate workers

2014-04-04 Thread Jan Algermissen
On 04 Apr 2014, at 11:18, prem yadav wrote: > Though cassandra can work but to me it looks like you could use a persistent > queue for example (rabbitMQ) to implement this. All your workers can > subscribe to a queue. > In fact, why not just MySQL? Hey, I have got a C* cluster that can (poten

Re: Using C* and CAS to coordinate workers

2014-04-04 Thread prem yadav
Though cassandra can work but to me it looks like you could use a persistent queue for example (rabbitMQ) to implement this. All your workers can subscribe to a queue. In fact, why not just MySQL? On Thu, Apr 3, 2014 at 11:44 PM, Jan Algermissen wrote: > Hi, > > maybe someone knows a nice solut

Using C* and CAS to coordinate workers

2014-04-03 Thread Jan Algermissen
Hi, maybe someone knows a nice solution to the following problem: I have N worker processes that are intentionally masterless and do not know about each other - they are stateless and independent instances of a given service system. These workers need to poll an event feed, say about every 10