Exactly. On Thu, Mar 9, 2017 at 7:40 AM, Greg Young <[email protected]> wrote:
> "You have to drop the generated events at some point if none of the > consumers can keep up, just need to figure out what is the best event > to drop." > > Unless we assume a near-infinite (larger than what we care about) > buffer such as how kafka, eventstore, or chronicle work. This does > require a switch from push to pull model subscriptions in most cases. > As example "give me the next 10 messages after #2929292". > > Cheers, > > Greg > > On Thu, Mar 9, 2017 at 3:35 PM, Max Kozlovsky <[email protected]> wrote: > > Hi, > > > > I think you are saying > > > > 1) you don't have to guarantee the delivery > > 2) You don't have control over the speed the events are generated > > 3) In event of some consumers not being able to keep up you want a best > > effort to deliver events to maximum consumers > > > > You have to drop the generated events at some point if none of the > consumers > > can keep up, just need to figure out what is the best event to drop. > > > > It seems a pull model might be better in this case. Maintain a queue of > > generated events in the producer of arbitrary finite size with a sequence > > number. Have consumers pull the events from producer starting from the > last > > sequence number they seen. If the queue becomes full, drop the oldest > event. > > This gives the best chance for maximum number of consumers to process the > > events at their own speed. > > > > Max > > > > On Thu, Mar 9, 2017 at 5:56 AM, Francesco <[email protected]> > > wrote: > >> > >> Hi Max, > >> > >> 2017-03-07 20:51 GMT+01:00 Max Kozlovsky <[email protected]>: > >> > I wonder what is the practical application of this model? If for > >> > correctness > >> > of the application it is enough to deliver every message to at least > one > >> > of > >> > the subscribers, why not always sent just one copy of a message to > just > >> > one > >> > of the subscribers? > >> No the correctness model is the following: assume that you have 10 > >> subscribers. Suddenly 1 of them looses connectivity / crashes / > >> whatever. The other 9 are ok. > >> What happens is that: > >> a) If I use the NODROP=1, my publisher blocks and stops processing its > >> own data, eventually leading to a drop of its own inputs. The whole > >> system is now blocked (publisher dropping its input events [not coming > >> from network], 9 subscribers all waiting for the publisher to > >> publish). > >> b) If I use a currently non-existing ZMQ_XPUB_ATLEASTONE=1 option, my > >> publisher keeps publishing, 9 subscribers keep working, just 1 is > >> stopped / out of service. Outcome in my case: 90% of my application is > >> still working, I have 10% of data that is not processed and lost. > >> > >> Which scenario is best? IMO it's b) :) > >> > >> > >> >Sends to the rest of the subscribers are overhead since > >> > they are not required for correctness? > >> They are required, but in case something goes bad I prefer the failure > >> of a part rather than the failure of everything. > >> > >> Francesco > >> _______________________________________________ > >> zeromq-dev mailing list > >> [email protected] > >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > _______________________________________________ > > zeromq-dev mailing list > > [email protected] > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > -- > Studying for the Turing test > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev >
_______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
