On Thu, 2018-10-18 at 15:05 +0000, [email protected] wrote: > I'm designing a multi-node pub-sub pattern (each node is a separate > computer running Debian Linux). The number of nodes is configurable > and can change over time (i.e. through configuration I can add or > remove nodes). I want to use XPUB-XSUB proxies on each node as > depicted below. The following link is for a picture showing the whole > topology: https://docs.google.com/drawings/d/1WZ-h_3cP1wc0WEEBmtrhH6C > DNakyNE4KTb-GApIV7Pk/edit?usp=sharing > > I wasn't sure if this mailing list allows adding links to external > pictures so here's an attempt at a drawing in plain text (hopefully > it doesn't get all messed up). > From Proxy 1 > From local PUBs on remote nodes > | | | > bind() connect() connect() > ipc:// tcp:// tcp:// > | | | > +-----+------+ +--+------+--+ > | XSUB | | XSUB | > +------------+ +------------+ > | Proxy 1 | | Proxy 2 | > +------------+ +------------+ > | XPUB | | XPUB | > +-----+------+ +-----+------+ > | | > bind() bind() > tcp:// ipc:// > | | > To Local SUBs To Local SUBs > and Proxy 2 on > remote nodes > The proxies use zmq_proxy_steerable(). I've already tested this and > it works. My question is regarding the XSUB socket on "Proxy 2". > That's the socket that connects to all the remote nodes. And since I > want to be able to add or remove nodes, how can I dynamically change > the connections on that socket? I thought of 3 ways I could > potentially do this. > Run zmq_proxy_steerable() in one thread (Thread1) and run a second > thread (Thread2) where I could invoke zmq_connect/disconnect to > change the connections. I suspect that it is probably not safe for > Thread2 to modify the connections while zmq_proxy_steerable() is > running in Thread1. > Thread2 could send a "PAUSE" to Thread1 to make it temporarily stop > processing messages. Then Thread2 invokes zmq_connect/disconnect to > change the connections. Finally, Thread2 sends "RESUME" to Thread1. > Again, not sure this is safe. Can the connections on a socket be > changed while zmq_proxy_steerable() is PAUSED? > Thread2 could write the list of nodes (IP addresses) to a data struct > shared with Thread1. Then Thread2 sends a TERMINATE to Thread1. > Thread1 terminates gracefully, reads the new list of nodes, changes > the connections on the socket accordingly, then goes back to running > zmq_proxy_steerable(). With this I'm not sure what would happen to > in-flight messages. That is, when TERMINATE is received by Thread1, > does it drop any of the messages it's currently processing? > What's the best approach? Is there another (safer) way to do this? > > Thanks, > Martin
PUB/SUB Sockets are not thread safe, so you'll need to terminate, do the changes you need, and restart the proxy. -- Kind regards, Luca Boccassi
signature.asc
Description: This is a digitally signed message part
_______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
