Re: Exploring Simply Queueing

2014-10-06 Thread Robert Coli
On Mon, Oct 6, 2014 at 1:40 PM, Jan Algermissen wrote: > Hmm, I was under the impression that issues with old queue state disappear > after gc_grace_seconds and that the goal primarily is to keep the rows > ‘short’ enough to achieve a tombstones read performance impact that one can > live with in

Re: Exploring Simply Queueing

2014-10-06 Thread Ranjib Dey
i want answer the first question why one might use cassandra as a queuing solution: - its the only opensource distributed persistence layer (i.e. no SPOF), that you can run over WAN and provide lan/wan specific quorum controls i know its sub optimal, as the deletion imposes additional compaction/r

Re: Exploring Simply Queueing

2014-10-06 Thread Jan Algermissen
Robert, On 06 Oct 2014, at 17:50, Robert Coli wrote: > In theory they can also be designed such that history is not infinite, which > mitigates the buildup of old queue state. > Hmm, I was under the impression that issues with old queue state disappear after gc_grace_seconds and that the goa

Re: Exploring Simply Queueing

2014-10-06 Thread Jan Algermissen
Shane, On 06 Oct 2014, at 16:34, Shane Hansen wrote: > Sorry if I'm hijacking the conversation, but why in the world would you want > to implement a queue on top of Cassandra? It seems like using a proper > queuing service > would make your life a lot easier. Agreed - however, the use case sim

Re: Exploring Simply Queueing

2014-10-06 Thread Robert Coli
On Mon, Oct 6, 2014 at 8:30 AM, Minh Do wrote: > Just let you know if you base your implementation on Netflix's queue > recipe, there are many issues with it. > > In general, we don't advise people to use that recipe so I suggest you to > save your time by not going that same route again. > I +1

Re: Exploring Simply Queueing

2014-10-06 Thread Minh Do
Hi Jan, Both Chris and Shane say what I believe the correct thinking. Just let you know if you base your implementation on Netflix's queue recipe, there are many issues with it. In general, we don't advise people to use that recipe so I suggest you to save your time by not going that same route

Re: Exploring Simply Queueing

2014-10-06 Thread Shane Hansen
Sorry if I'm hijacking the conversation, but why in the world would you want to implement a queue on top of Cassandra? It seems like using a proper queuing service would make your life a lot easier. That being said, there might be a better way to play to the strengths of C*. Ideally everything you

Re: Exploring Simply Queueing

2014-10-05 Thread Jan Algermissen
Chris, thanks for taking a look. On 06 Oct 2014, at 04:44, Chris Lohfink wrote: > It appears you are aware of the tombstones affect that leads people to label > this an anti-pattern. Without "due" or any time based value being part of > the partition key means you will still get a lot of bui

Re: Exploring Simply Queueing

2014-10-05 Thread Chris Lohfink
It appears you are aware of the tombstones affect that leads people to label this an anti-pattern. Without "due" or any time based value being part of the partition key means you will still get a lot of buildup. You only have 1 partition per shard which just linearly decreases the tombstones.

Exploring Simply Queueing

2014-10-05 Thread Jan Algermissen
Hi, I have put together some thoughts on realizing simple queues with Cassandra. https://github.com/algermissen/cassandra-ruby-queue The design is inspired by (the much more sophisticated) Netfilx approach[1] but very reduced. Given that I am still a C* newbie, I’d be very glad to hear some th