And what if there are no subscribers? Publishing should block?
It sounds like your goal is some kind of sensible flow control (I'm not sure what else an "at least 1 subscriber can read" use-case really is). My advice is to publish at a fixed rate, for example by sleeping between publishes. I do wish ZMQ_RATE worked for all PUB sockets and not just PGM-based. On Fri, Mar 3, 2017, at 12:31 PM, Francesco wrote: > Hi all, > > I'm new to ZeroMQ but so far all my experiments with it have been > quite positive, so thanks for your great work! > > However, one thing that I cannot really sort out is the following: I > have written a small sample program where I create a ZMQ_PUB socket > and I continuously zmq_msg_send() messages in it, never sleeping. This > is simulating what I will do in a larger program where I plan to use > ZeroMQ. Such program is a massively-parallel utility that needs to > send in a fanout fashion (1 to many) several short messages per second > (up to say 1million messages / second). > > In my scenario I don't care about subscribers joining late or > eventually loosing some messages (if the network is slow or the > subscriber itself is too slow). I do care however about detecting such > conditions where messages are dropped. > > My problem is the "fast producer" one: this sample program shows that > the zmq_msg_send() never returns an error; doing some math such my > little utility says that is publishing data at rates up to 400Gbps... > the only problem is that the NIC on the computer is a 1Gbps NIC. Of > course all subscribers report missing 99% of messages (I put a > sequence number in the messages I send). > > This problem does not appear to be new, some interesting references I > found are: > > > http://grokbase.com/t/zeromq/zeromq-dev/11ca11s9b9/pub-sub-pattern-rate-control-and-backpressure > (Dec 2011) > > http://grokbase.com/t/zeromq/zeromq-dev/12797gy703/notify-send-er-that-theyve-hit-the-high-water-mark > (Jul 2012) > > http://grokbase.com/t/zeromq/zeromq-dev/129n2e2sx5/high-water-mark-notification-for-publisher > (Sep 2012) > > I experimented a little bit with ZeroMQ and found that: > - ZMQ_RATE is not doing anything > - ZMQ_XPUB_NODROP set to 1 works BUT basically ties the publisher to > the SLOWEST subscriber: from my experiments it looks like (despite > the documentation) the publisher now blocks every time there is > just 1 subscriber queue that has hit the HWM > > as written in some of these posts what I would like to have is a > NODROP socket option that allows my publisher to send data as fast as > the FASTEST subscriber can handle. > > So here's my question: is there any way to achieve the above, i.e. > block the publisher if ALL subscriber queues hit their HWM ? > > > Thanks a lot for any hint! > > Francesco > > > > > > _________________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
