Hello gurus, Thank's very much for your answers. Here are some more info related to the case I am debugging. As I said, we are using a multipart message with 4 parts. On the subscriber side, I have:
Message part 1: Size = 41 bytes - Data ptr = xxxa986 -- zmq_msg_t instance adr = xxx9c90 Message part 2: Size = 1 bytes - Data ptr = xxx9cd8 -- zmq_msg_t instance adr = xxx9cd0 Message part 3: Size = 15 bytes - Data ptr = xxx9d18 -- zmq_msg_t instance adr = xxx9d10 Message part 4: Size = 44 bytes - Data ptr = xxxa9e3 --zmq_msg_t instance adr == xxx9d50 From my understanding of what happens, it seems that part 2 and 3 are "vsm" and we can see that their buffer is part of the zmq_msg_t instances. But parts 1 and 4 (size 41 or 44 bytes) are not "vsm" and their buffers are in other memory chunk that their zmq_msg_t instances. But those two buffers are not aligned. This generates troubles in our buffer decoding system! From Jens answer, I understand that this is related to changes done for release 4.2 Is there a way to retrieve aligned buffers? Sincerely Emmanuel On 11/14/2016 08:34 PM, Jens Auer wrote: > Hi, > > I think I have an idea why you are seeing unaligned messages, but this only > applies to messages where the payload is not stored in the msg_t object > itself. I think the threshold for this is 64 bytes. In ZeroMQ 4.1, receiving > messages was done by first receiving from the socket into a static 8kb > buffer, and then a new message object was created that allocated memory > externally by calling malloc. The payload was then copied from the receive > buffer to the message buffer. The malloced message buffer was aligned > probably. > > In ZeroMQ 4.2, this is changed to reduce the number of malloc calls and copy > operations. The receive buffer is now dynamically allocated as a 8kb block, > and messages are constructed as zero-copy messages using the part of the > receive buffer containing the payload. This saves malloc calls and copy > operations and increases performance. However, the payload may now start at > basically arbitrary addresses. As an example, let's assume that we receive a > small message of 10 bytes and a large message of 1kb, both received in a > single call to recv on the socket. The engine allocates a new buffer of 8kb, > calls recv(socket, buffer) and the data is written to the buffer. A small > message is then created which contains the data from byte 2-11 in the msg_t, > byte 1 contains the header. At byte 12 starts the header of the next message, > and at byte 22(?) starts the payload. The large message is created as a > zero-copy message using the pointer to byte 22 as storage. This is not > aligned to a 4-byte address. > > Could you provide some more information about the sizes of the messages that > you receive? How do you decode the buffer content? > > Best wishes, > Jens > > -----Ursprüngliche Nachricht----- > Von: zeromq-dev [mailto:[email protected]] Im Auftrag von > Emmanuel Taurel > Gesendet: Montag, 14. November 2016 16:49 > An: [email protected] > Betreff: [zeromq-dev] Zmq 4.2.0 aligned memory > > Hello all, > > We are using zeromq since years now without troubles. We have recently tried > our software using Zmq 4.2.0 (on linux hosts). > For our application, we are using multipart messages with 4 parts in > publish/subscribe mode. > With Zmq 4.0.5, on the subscriber side, when we get the last message part, > the received buffer was memory aligned (at least on 0x4 border). > Unfortunately, with Zmq 4.2.0, the buffer is not aligned any more. > For instance with Zmq 4.0.5, the buffer was at address xxx08 while with Zmq > 4.2.0, it is at address xxx23. > > I don't know if it is relevant but our messages are relatively small messages > (few tens of bytes) This is a problem for us in decoding the buffer content. > > Is there something to be done to have memory aligned buffers? > > Thank's in advance for your answers > > Emmanuel > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > --- > Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. > https://www.avast.com/antivirus > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
