It's quite normal to use a different protocol for inproc work, in certain cases at least. Passing pointers is an effective way to pass a whole structure. We do this e.g. in Malamute for passing messages between actors. There is no intention of ever making actors into remote objects. So a custom inproc protocol is fair.
When you use inproc to simulate discrete components, then you would not do this. On Sat, Jan 16, 2016 at 10:42 PM, Wenzel Jakob <[email protected]> wrote: > Dear Charles, > > note that that is quite different though. If I have to explicitly send > pointers via ZeroMQ, then it means that INPROC communication uses a > specialized protocol that would not work with external hosts. > > I just ran some experiments to test the hypothesis myself, and it looks like > ZeroMQ delivers zero copies through the entire chain of operations. On my > machine, I get the following output when transferring data at a certain > memory location through INPROC transport (program attached). > > Sending message for data at 0x107726ef0 > Received message which points to data at 0x107726ef0 > free_fn called for data at 0x107726ef0 > > In other words, the address stays the same — no copies. :) > > Best, > Wenzel > > PS: Here is the program I used to test this. > > > #include <zmq.h> > > #include <stdio.h> > > #include <string.h> > > #include <assert.h> > > > > const char *msg_content = "A message"; > > void free_fn(void *data, void *hint) { > > printf("free_fn called for data at %p\n", data); > > } > > int main(void) { > > int rc; > > void *ctx = zmq_ctx_new(); > > void *rep_socket = zmq_socket(ctx, ZMQ_REP); > > void *req_socket = zmq_socket(ctx, ZMQ_REQ); > > zmq_msg_t msg_out, msg_in; > > > > rc = zmq_bind(rep_socket, "inproc://test"); assert(rc == 0); > > rc = zmq_connect(req_socket, "inproc://test"); assert(rc == 0); > > printf("Sending message for data at %p\n", msg_content); > > rc = zmq_msg_init_data(&msg_out, (void *) msg_content, > strlen(msg_content), free_fn, NULL); assert(rc == 0); > > rc = zmq_msg_send(&msg_out, req_socket, 0); assert(rc != -1); > > zmq_msg_close(&msg_out); > > > > rc = zmq_msg_init(&msg_in); assert (rc == 0); > > rc = zmq_msg_recv(&msg_in, rep_socket, 0); assert(rc != -1); > > void *ptr = zmq_msg_data(&msg_in); > > printf("Received message which points to data at %p\n", ptr); > > zmq_msg_close(&msg_in); > > zmq_close(rep_socket); > > zmq_close(req_socket); > > zmq_ctx_term(ctx); > > return 0; > > } > > > > > On Jan 16, 2016, at 7:54 PM, Brian T. Carcich <[email protected]> > wrote: > > if all threads are in the same process, just use zmq to pass the pointer; > that guarantees ZMQ will not duplicates the content of the message. > > of course, then you have to be very disciplined about making sure only one > thread ever frees the memory, but that should not be too hard. > > On Sat, Jan 16, 2016 at 11:25 AM, Wenzel Jakob <[email protected]> wrote: >> >> Dear Charles, >> >> this page notes that the creation of the message itself is guaranteed not >> to cause a copy operation. >> >> It does not, however, say anything about moving the message from one >> socket to the other, and about the behavior of zmq_recvmsg. >> >> (My question is thus if the zero-copy guarantee applies to the whole chain >> of operations.) >> >> Thank you, >> Wenzel >> >> On Jan 16, 2016, at 5:20 PM, Charles Remes <[email protected]> wrote: >> >> Look at this man page: >> >> http://api.zeromq.org/3-2:zmq-msg-init-data >> >> Zeromq does zero copy. >> >> However, before you go and write a bunch of complex code, just use the >> normal facilities first and benchmark it. You may not need to do this extra >> work. >> >> >> On Jan 16, 2016, at 10:13, Wenzel Jakob <[email protected]> wrote: >> >> Hello all, >> >> I’m interested in using ZeroMQ to build a multithreaded application which >> uses INPROC transport to propagate messages between threads. >> >> My plan was to send messages using zmq_sendmsg (created using >> zmq_msg_init_data with a custom deallocator), and to receive messages, I was >> planning to use zmq_recvmsg. >> >> I’m concerned if INPROC will perform unnecessary copies of messages, which >> can be very large in my application (i.e. hundreds of megabytes). In >> principle, it should just be possible to do it without copies by just moving >> zmq_msg_t* from the inproc socket of one thread to that of the other — >> however, I am not familiar enough with the implementation of ZeroMQ to know >> if that is indeed what happens. Can anyone here advise? >> >> Thank you in advance for your help! >> >> Best regards, >> Wenzel >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
