Dear Charles,
note that that is quite different though. If I have to explicitly send pointers
via ZeroMQ, then it means that INPROC communication uses a specialized protocol
that would not work with external hosts.
I just ran some experiments to test the hypothesis myself, and it looks like
ZeroMQ delivers zero copies through the entire chain of operations. On my
machine, I get the following output when transferring data at a certain memory
location through INPROC transport (program attached).
Sending message for data at 0x107726ef0
Received message which points to data at 0x107726ef0
free_fn called for data at 0x107726ef0
In other words, the address stays the same — no copies. :)
Best,
Wenzel
PS: Here is the program I used to test this.
#include <zmq.h>
#include <stdio.h>
#include <string.h>
#include <assert.h>
const char *msg_content = "A message";
void free_fn(void *data, void *hint) {
printf("free_fn called for data at %p\n", data);
}
int main(void) {
int rc;
void *ctx = zmq_ctx_new();
void *rep_socket = zmq_socket(ctx, ZMQ_REP);
void *req_socket = zmq_socket(ctx, ZMQ_REQ);
zmq_msg_t msg_out, msg_in;
rc = zmq_bind(rep_socket, "inproc://test"); assert(rc == 0);
rc = zmq_connect(req_socket, "inproc://test"); assert(rc == 0);
printf("Sending message for data at %p\n", msg_content);
rc = zmq_msg_init_data(&msg_out, (void *) msg_content, strlen(msg_content),
free_fn, NULL); assert(rc == 0);
rc = zmq_msg_send(&msg_out, req_socket, 0); assert(rc != -1);
zmq_msg_close(&msg_out);
rc = zmq_msg_init(&msg_in); assert (rc == 0);
rc = zmq_msg_recv(&msg_in, rep_socket, 0); assert(rc != -1);
void *ptr = zmq_msg_data(&msg_in);
printf("Received message which points to data at %p\n", ptr);
zmq_msg_close(&msg_in);
zmq_close(rep_socket);
zmq_close(req_socket);
zmq_ctx_term(ctx);
return 0;
}
> On Jan 16, 2016, at 7:54 PM, Brian T. Carcich <[email protected]> wrote:
>
> if all threads are in the same process, just use zmq to pass the pointer;
> that guarantees ZMQ will not duplicates the content of the message.
>
> of course, then you have to be very disciplined about making sure only one
> thread ever frees the memory, but that should not be too hard.
>
> On Sat, Jan 16, 2016 at 11:25 AM, Wenzel Jakob <[email protected]
> <mailto:[email protected]>> wrote:
> Dear Charles,
>
> this page notes that the creation of the message itself is guaranteed not to
> cause a copy operation.
>
> It does not, however, say anything about moving the message from one socket
> to the other, and about the behavior of zmq_recvmsg.
>
> (My question is thus if the zero-copy guarantee applies to the whole chain of
> operations.)
>
> Thank you,
> Wenzel
>
>> On Jan 16, 2016, at 5:20 PM, Charles Remes <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Look at this man page:
>>
>> http://api.zeromq.org/3-2:zmq-msg-init-data
>> <http://api.zeromq.org/3-2:zmq-msg-init-data>
>>
>> Zeromq does zero copy.
>>
>> However, before you go and write a bunch of complex code, just use the
>> normal facilities first and benchmark it. You may not need to do this extra
>> work.
>>
>>
>>> On Jan 16, 2016, at 10:13, Wenzel Jakob <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>> Hello all,
>>>
>>> I’m interested in using ZeroMQ to build a multithreaded application which
>>> uses INPROC transport to propagate messages between threads.
>>>
>>> My plan was to send messages using zmq_sendmsg (created using
>>> zmq_msg_init_data with a custom deallocator), and to receive messages, I
>>> was planning to use zmq_recvmsg.
>>>
>>> I’m concerned if INPROC will perform unnecessary copies of messages, which
>>> can be very large in my application (i.e. hundreds of megabytes). In
>>> principle, it should just be possible to do it without copies by just
>>> moving zmq_msg_t* from the inproc socket of one thread to that of the other
>>> — however, I am not familiar enough with the implementation of ZeroMQ to
>>> know if that is indeed what happens. Can anyone here advise?
>>>
>>> Thank you in advance for your help!
>>>
>>> Best regards,
>>> Wenzel
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> <http://lists.zeromq.org/mailman/listinfo/zeromq-dev>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> [email protected] <mailto:[email protected]>
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> <http://lists.zeromq.org/mailman/listinfo/zeromq-dev>
>
>
> _______________________________________________
> zeromq-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> <http://lists.zeromq.org/mailman/listinfo/zeromq-dev>
>
>
> _______________________________________________
> zeromq-dev mailing list
> [email protected]
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
