Alright, thanks everyone, really useful info. I will rethink my design to eliminate the need for sharing the sockets between threads :).
On Sun, May 13, 2018 at 1:03 AM Justin Karneges <[email protected]> wrote: > The real issue here is thread affinity. Mere thread-unsafety can always be > worked-around by the application developer, by ensuring that no two threads > ever access the same resource at the same time. It is my understanding that > libzmq sockets are not thread safe, but they also don't have thread > affinity, so you can access a single socket from different threads if you > handle safety on your own. > > Personally, I agree with the libzmq authors in that trying to do I/O on > the same socket from multiple threads is an anti-pattern. However, I > consider certain kinds of sharing to be legitimate, for example creating a > socket in one thread and then handing it off to a second thread, and only > that second thread does I/O. I think that fits within the spirit of libzmq, > since only one thread is actually doing I/O on the socket. If you find > yourself doing I/O on a socket from multiple threads, protecting the access > with a mutex, then you're in anti-pattern territory. That said, it should > work just fine, and libzmq authors' opinions won't stop you. :) > > FWIW, you might run into thread affinity in other libraries. For example, > last I checked, PyZMQ has thread-affinity of sockets. If you create a > socket in one thread and hand it off to another, I/O from the other thread > will fail. But it is my understanding that libzmq itself does not have such > limitation. > > On Sat, May 12, 2018, at 2:08 PM, Bill Torpey wrote: > > 1. FWIW, my personal experience has been that it is safe to share a > ZMQ_PUB socket between threads if protected with a mutex. Basically the > sequence is lock/send/unlock. > > The alternative is to have an internal zmq_proxy with inproc on one side > and something like TCP on the other side, which is an extra send/receive > hop. I have not found that to be necessary, and I have done a *lot* of > testing (which has found several other edge cases). > > FWIW the docs specifically bless this approach (http://zeromq.org/area:faq > ): > > For those situations where a dedicated socket per thread is infeasible, a > socket may be shared *if and only if* each thread executes a full memory > barrier before accessing the socket. Most languages support a Mutex or > Spinlock which will execute the full memory barrier on your behalf. > > and here (http://zguide.zeromq.org/page:all#Multithreading-with-ZeroMQ): > > Technically it's possible to migrate a socket from one thread to another > but it demands skill. > > > I have not tried doing that with ZMQ_SUB sockets — there is really no > point, as a sigle thread sitting in a zmq_poll call on multiple sockets is > probably the cleanest design anyway. > > > 2. I can definitely testify that trying to share a ZMQ_SUB socket > *without* mutexes is guaranteed to crash, and that trying to share a > ZMQ_SUB socket *with* mutexes is almost impossible to do without > deadlocking. > > > 3. Another situation is to have a single “main” thread setup all the > sockets at startup and then hand them off to other threads (and never touch > them again). Strictly speaking you may only need a memory barrier, but > using a mutex in this case ensures that you don’t try to use the socket > concurrently from more than one thread. > > > 4. Last but not least, if you’re planning on using inproc sockets for > inter-thread communication, you may find this helpful: > https://github.com/WallStProg/zmqtests/tree/master/threads > > > On May 8, 2018, at 3:58 PM, Thomas Rodgers <[email protected]> wrote: > > No, not really. I have designed my zmq based systems to either - > * orchestrate connections over the inproc transport > * performed the delicate dance of explicitly passing a fresh socket to a > fresh thread the way that czmq actors > <http://czmq.zeromq.org/manual:zactor> work, > as implemented in azmq > <https://github.com/zeromq/azmq/blob/master/azmq/detail/actor_service.hpp> > > On Mon, May 7, 2018 at 11:58 AM, Attila Magyari <[email protected]> wrote: > > This sounds promising. My concern is if there are some background threads > internally in the ZMQ library, which would use the socket out of sync with > mine, or anything else that I'm not thinking about. I can ensure that the > socket won't be used from multiple threads. > > Do you have any experience with this, or only theory? > > Thank you, > Attila > > > On Mon, May 7, 2018 at 5:49 PM Thomas Rodgers <[email protected]> > wrote: > > The zmq docs say you need to ensure a ‘full fence’ a mutex is one way of > achieving that. > > If you are using C++ (-std=c++11 or later) and you can guarantee there is > no racy usage from another thread, you can issue a memory fence using - > > std::atomic_thread_fence > > > On Mon, May 7, 2018 at 8:42 AM Attila Magyari <[email protected]> wrote: > > Hello, > > I know that zmq sockets are not thread safe. However I can't think of a > good design (explained below) without using the socket in several threads. > What I want to do is connect the socket in a thread, and then use it in a > different one until the end, and never in parallel between several threads. > I will guard the access with a mutex as well. Do you think this will be > safe? > > Just in case someone might have a better idea for my design, here is what > I'm trying to do: > I have an application which starts another process, passes a port to the > newly created process, so it can connect back to the main application > through a ZMQ_PAIR socket. Both processes are local to the machine. The > main application will request from the second one, but the second one can > initiate messages without an explicit request as well. Do you have a nicer > design to achieve something like this? > > Thank you in advance! > > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > *_______________________________________________* > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev >
_______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
