Nick, I think it’s best to think of multipart messages as a serialization technique (like fields in Protocol Buffers) rather than as separate messages. Because they are handled atomically at the API layer, a multipart message is an all-or-nothing thing. So rather than worrying about multipart or not in thinking about your design, just consider “messages” as each quantum of communication between your nodes (which may be a ZMQ multipart message).
Your testing demonstrates expected behavior to me. With a very low high-water mark and a high data rate, you should expect to hit the limit. As the Socket API <https://zeromq.org/socket-api/#high-water-mark> says, > If this limit has been reached the socket enters an exceptional state and > depending on the socket type, ZeroMQ will take appropriate action such as > blocking or dropping sent messages. Refer to the individual socket > descriptions below for details on the exact action taken for each socket type. In your case, the “appropriate action” was dropping a message. Stephen’s explanation of packet management was very good, and should help inform your design. You either need appropriately sized buffers, or some sort of flow-control technique. — Pragmatics must take precedence over elegance For Nature is not impressed Jim Melton, 303-829-0447 http://blogs.melton.space/pharisee/ > On May 23, 2025, at 21:39, Stephen Gray <[email protected]> wrote: > > Hi. I'm reading this on the context of having worked with deeply embedded > message routing in closed but WAN packet systems (as opposed to TDM) in > subsea. > > What I see in your problem description are the essential elements of a > problem I encountered before. > Our routing nodes were FPGA based and design constrained by ultra low power > limits. > Where in the previous TDM solution, all packets had an allotted time, the > temporal packet arrival rate and internal was predictable and stable, when we > moved to a packet solution we then had random peak traffic rates to contend > with even though the average traffic rate had not increased. > The solution now had to have larger buffers in each node. In order not to > lose any packets, which was a requirement of the system, we needed large > enough buffers to also handle the statistically unlikely highest peak rates, > or else we observed actual packet loss. > We looked at mechanisms which would evaluate traffic arrival density, and > take local or back propagated action to previous nodes to throttle back, but > we found that there would still be rare cases where we would detect buffer > overflow, but then we would have lost at least one packet already. > > It's a conceptual view .. but by experimentation we ended up with handling > the average traffic rate by throttling gradually when the peak arrival rate > exceeded say 80% capacity, essentially relying on buffer space in preceding > routing nodes, and handling the statistically rare losses with a retry > mechanism. > > As I seen it, if there is a requirement for zero data loss, then a a system > of nodes with asynchronous send patterns need much much more buffer space. > This is where routing mechanisms like ATM had advantages. > > We had got too far in development to go back to the TDM solution .. but the > optimal solution would have been TDM for the isochronous traffic and a > residual buffer system for asynchronous packets in the remaining intervals. > > Hope this perspective .. while a little general .. may help. > > Best Regards > Stephen > > > 23. mai 2025 kl. 19:26 skrev Nicholas Long via zeromq-dev > <[email protected]>: > > > Hi, > > I am new to zmq, so this may be a bit of a dumb question. > > I am using a dealer-router, where the dealer is sending a-lot of multipart > messages to the router. It seems like if I have some weird processing pattern > on the router, where I do not service the messages correctly, then I get > dropped multipart messages. > > This is admittedly intermittent, and goes away if I increase the receive high > water mark on the router. But I am able to reliably reproduce if I send > multipart messages (a group of 3), and have the receive high water mark of 10. > > So it seems like there is some case where the router's zmq will silently drop > multipart messages when there is not enough room for the whole message in the > internal buffers. This is counter to my expectation that when there is not > enough on zmq internal buffers, the copy from the socket to the zmq internal > buffer would block, this blocking then propagates down back across the tcp > socket, and then causes the dealer's send function to block. > > Am I missing something? > > Thanks, > Nick > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
