The messages structures themselves are always on the stack. The TCP receive is batched, and if there are multiple messages in an 8KB kernel buffer, each message's content_t simply points to the right place for the data in that shared buffer, which is refcounted. The content_t structure is also in the same memory zone, which is split to allow enough content_t for 8KB/minimum_size_msg+1 messages - so in practice there is one allocation on ~19KB which is shared with as many messages as their data can fit in 8KB that are received in one TCP read. On Fri, 2019-08-16 at 16:46 +0200, Francesco wrote: > Hi Doron,Ok the zmq_msg_init_allocator approach looks fine to me. I > hope I have time to work on that in the next couple of weeks (unless > someone else wants to step in of course :-) ). > > > Anyway the current approach works for sending messages...I wonder how > the Rx side works and if we could exploit memory pooling also for > that... Is there any kind of documentation on how the engine works > for Rx (or some email thread) perhaps? > > I know there is some zero copy mechanism in place but it's not > totally clear to me: is the zmq_msg_t coming out of zmq API pointing > directly to the kernel buffers? > Thanks > Francesco > > > Il gio 15 ago 2019, 11:39 Doron Somech <[email protected]> ha > scritto: > > maybe zmq_msg_init_allocator which accept the allocator. > > > > With that pattern we do need the release method, the zmq_msg will > > handle it internally and register the release method as the free > > method of the zmq_msg. They do need to have the same signature. > > On Thu, Aug 15, 2019 at 12:35 PM Francesco < > > [email protected]> wrote: > > > Hi Doron, hi Jens,Yes the allocator method is a nice solution. > > > I think it would be nice to have libzmq provide also a memory > > > pool implementation but use as default the malloc/free > > > implementation for backward compatibility. > > > > > > It's also important to have a smart allocator that internally > > > contains not just one but several pools for different packet > > > size classes,to avoid memory waste. But I think this can fit > > > easily in the allocator pattern sketched out by Jens. > > > > > > Btw another issue unrelated to the allocator API but regarding > > > performance aspects: I think it's important to avoid not only the > > > msg buffer but also the allocation of the content_t structure and > > > indeed in my preliminary merge request I did modify zmq_msg_t of > > > type_lmsg to use the first 40b inside the pooled buffer. > > > Of course this approach is not backward compatible with the > > > _init_data() semantics. > > > How do you think this would best be approached? > > > I guess we may have a new _init_data_and_controlblock() helper > > > that does the trick of taking the first 40bytes of the provided > > > buffer? > > > > > > Thanks > > > Francesco > > > > > > > > > Il mer 14 ago 2019, 22:23 Doron Somech <[email protected]> ha > > > scritto: > > > > Jens I like the idea. > > > > > > > > We actually don't need the release method. > > > > The signature of the allocate should receive zmq_msg and > > > > allocate it. > > > > > > > > int (&allocate)(zmq_msg *msg, size_t size, void *obj); > > > > > > > > When the allocator will create the zmq_msg it will provide the > > > > release method to the zmq_msg in the constructor. > > > > > > > > This is important in order to forward messages between sockets, > > > > so the release method is part of the msg. This is already > > > > supported by zmq_msg which accept free method with a hint (obj > > > > in your example). > > > > > > > > The return value of allocate will be success indication, like > > > > the rest of zeromq methods. > > > > > > > > zeromq actually already support pool mechanism when sending, > > > > using zmq_msg api. Receiving is the problem, your suggestion > > > > solve it nicely. > > > > > > > > By the way, memory pool already supported in NetMQ in a very > > > > similar solution as you suggested. (It is global for all > > > > sockets without override) > > > > > > > > > > > > > > > > On Wed, Aug 14, 2019, 22:41 Jens Auer < > > > > [email protected]> wrote: > > > > > Hi, > > > > > Maybe this can be combined with a request that I have seen a > > > > > couple of times to be able to configure the allocator used in > > > > > libzmq? I am thinking of something like > > > > > > > > > > struct zmq_allocator { > > > > > void* obj; > > > > > void* (&allocate)(size_t n, void* obj); > > > > > void (&release)(void* ptr, void* obj); > > > > > }; > > > > > > > > > > void* useMalloc(size_t n, void*) {return malloc(n);} > > > > > void freeMalloc(void* ptr) {free(ptr);} > > > > > > > > > > zmq_allocator& zmg_default_allocator() { > > > > > static zmg_allocator defaultAllocator = {nullptr, > > > > > useMalloc, freeMalloc}; > > > > > Return defaultAllocator; > > > > > } > > > > > > > > > > The context could then store the allocator for libzmq, and > > > > > users could set a specific allocator as a context option, > > > > > e.g. with a zmq_ctx_set. A socket created for a context can > > > > > then inherit the default allocator or set a special allocator > > > > > as a socket option. > > > > > > > > > > class MemoryPool {…}; // hopefully thread-safe > > > > > void* poolAllocate(size_t n) {return > > > > > > > > > > MemoryPool pool; > > > > > > > > > > void* allocatePool(size_t n, void* pool) {return > > > > > static_cast<MemoryPool*>(pool)->allocate(n);} > > > > > void releasePool(void* ptr, void* pool) > > > > > {static_cast<MemoryPool*>(pool)->release(ptr);} > > > > > > > > > > zmq_allocator pooledAllocator { > > > > > &pool, allocatePool, releasePool > > > > > } > > > > > > > > > > void* cdx = zmq_ctx_new(); > > > > > zmq_ctx_set(ZMQ_ALLOCATOR, &pooledAllocator); > > > > > > > > > > Cheers, > > > > > Jens > > > > > > > > > > > Am 13.08.2019 um 13:24 schrieb Francesco < > > > > > > [email protected]>: > > > > > > > > > > > > Hi all, > > > > > > > > > > > > today I've taken some time to attempt building a memory- > > > > > > pooling > > > > > > mechanism in ZMQ local_thr/remote_thr benchmarking > > > > > > utilities. > > > > > > Here's the result: > > > > > > https://github.com/zeromq/libzmq/pull/3631 > > > > > > This PR is a work in progress and is a simple modification > > > > > > to show the > > > > > > effects of avoiding malloc/free when creating zmq_msg_t > > > > > > with the > > > > > > standard benchmark utils of ZMQ. > > > > > > > > > > > > In particular the very fast, zero-lock, > > > > > > single-producer/single-consumer queue from: > > > > > > https://github.com/cameron314/readerwriterqueue > > > > > > is used to maintain between the "remote_thr" main thread > > > > > > and its ZMQ > > > > > > background IO thread a list of free buffers that can be > > > > > > used. > > > > > > > > > > > > Here are the graphical results: > > > > > > with mallocs / no memory pool: > > > > > > > > > > > > https://cdn1.imggmi.com/uploads/2019/8/13/9f009b91df394fa945cd2519fd993f50-full.png > > > > > > with memory pool: > > > > > > > > > > > > https://cdn1.imggmi.com/uploads/2019/8/13/f3ae0d6d58e9721b63129c23fe7347a6-full.png > > > > > > > > > > > > Doing the math the memory pooled approach shows: > > > > > > > > > > > > mostly the same performances for messages <= 32B > > > > > > +15% pps/throughput increase @ 64B, > > > > > > +60% pps/throughput increase @ 128B, > > > > > > +70% pps/throughput increase @ 210B > > > > > > > > > > > > [the tests were stopped at 210B because my current quick- > > > > > > dirty memory > > > > > > pool approach has fixed max msg size of about 210B]. > > > > > > > > > > > > Honestly this is not a huge speedup, even if still > > > > > > interesting. > > > > > > Indeed with these changes the performances now seem to be > > > > > > bounded by > > > > > > the "local_thr" side and not by the "remote_thr" anymore. > > > > > > Indeed the > > > > > > zmq background IO thread for local_thr is the only thread > > > > > > at 100% in > > > > > > the 2 systems and its "perf top" now shows: > > > > > > > > > > > > 15,02% libzmq.so.5.2.3 [.] zmq::metadata_t::add_ref > > > > > > 14,91% libzmq.so.5.2.3 [.] > > > > > > zmq::v2_decoder_t::size_ready > > > > > > 8,94% libzmq.so.5.2.3 [.] zmq::ypipe_t<zmq::msg_t, > > > > > > 256>::write > > > > > > 6,97% libzmq.so.5.2.3 [.] zmq::msg_t::close > > > > > > 5,48% libzmq.so.5.2.3 [.] > > > > > > zmq::decoder_base_t<zmq::v2_decoder_t, > > > > > > zmq::shared_message_memory_allo > > > > > > 5,40% libzmq.so.5.2.3 [.] zmq::pipe_t::write > > > > > > 4,94% libzmq.so.5.2.3 [.] > > > > > > zmq::shared_message_memory_allocator::inc_ref > > > > > > 2,59% libzmq.so.5.2.3 [.] > > > > > > zmq::msg_t::init_external_storage > > > > > > 1,63% [kernel] [k] > > > > > > copy_user_enhanced_fast_string > > > > > > 1,56% libzmq.so.5.2.3 [.] zmq::msg_t::data > > > > > > 1,43% libzmq.so.5.2.3 [.] zmq::msg_t::init > > > > > > 1,34% libzmq.so.5.2.3 [.] zmq::pipe_t::check_write > > > > > > 1,24% libzmq.so.5.2.3 [.] > > > > > > zmq::stream_engine_base_t::in_event_internal > > > > > > 1,24% libzmq.so.5.2.3 [.] zmq::msg_t::size > > > > > > > > > > > > Do you know what this stacktrace might mean? > > > > > > I would expect to have that ZMQ background thread topping > > > > > > in its > > > > > > read() system call (from TCP socket)... > > > > > > > > > > > > Thanks, > > > > > > Francesco > > > > > > > > > > > > > > > > > > Il giorno ven 19 lug 2019 alle ore 18:15 Francesco > > > > > > <[email protected]> ha scritto: > > > > > > > Hi Yan, > > > > > > > Unfortunately I have interrupted my attempts in this area > > > > > > > after getting some strange results (possibly due to the > > > > > > > fact that I tried in a complex application context... I > > > > > > > should probably try hacking a simple zeromq example > > > > > > > instead!). > > > > > > > > > > > > > > I'm also a bit surprised that nobody has tried and posted > > > > > > > online a way to achieve something similar (Memory pool > > > > > > > zmq send) ... But anyway It remains in my plans to try > > > > > > > that out when I have a bit more spare time... > > > > > > > If you manage to have some results earlier, I would be > > > > > > > eager to know :-) > > > > > > > > > > > > > > Francesco > > > > > > > > > > > > > > > > > > > > > Il ven 19 lug 2019, 04:02 Yan, Liming (NSB - CN/Hangzhou) > > > > > > > <[email protected]> ha scritto: > > > > > > > > Hi, Francesco > > > > > > > > Could you please share the final solution and > > > > > > > > benchmark result for plan 2? Big Thanks. > > > > > > > > I'm concerning this because I had tried the similar > > > > > > > > before with zmq_msg_init_data() and zmq_msg_send() but > > > > > > > > failed because of two issues. 1) My process is > > > > > > > > running in background for long time and finally I found > > > > > > > > it occupies more and more memory, until it exhausted > > > > > > > > the system memory. It seems there's memory leak with > > > > > > > > this way. 2) I provided *ffn for deallocation but the > > > > > > > > memory freed back is much slower than consumer. So > > > > > > > > finally my own customized pool could also be exhausted. > > > > > > > > How do you solve this? > > > > > > > > I had to turn back to use zmq_send(). I know it has > > > > > > > > memory copy penalty but it's the easiest and most > > > > > > > > stable way to send message. I'm still using 0MQ 4.1.x. > > > > > > > > Thanks. > > > > > > > > > > > > > > > > BR > > > > > > > > Yan Limin > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: zeromq-dev [mailto: > > > > > > > > [email protected]] On Behalf Of Luca > > > > > > > > Boccassi > > > > > > > > Sent: Friday, July 05, 2019 4:58 PM > > > > > > > > To: ZeroMQ development list < > > > > > > > > [email protected]> > > > > > > > > Subject: Re: [zeromq-dev] Memory pool for zmq_msg_t > > > > > > > > > > > > > > > > There's no need to change the source for experimenting, > > > > > > > > you can just use _init_data without a callback and with > > > > > > > > a callback (yes the first case will leak memory but > > > > > > > > it's just a test), and measure the difference between > > > > > > > > the two cases. You can then immediately see if it's > > > > > > > > worth pursuing further optimisations or not. > > > > > > > > > > > > > > > > _external_storage is an implementation detail, and it's > > > > > > > > non-shared because it's used in the receive case only, > > > > > > > > as it's used with a reference to the TCP buffer used in > > > > > > > > the system call for zero-copy receives. Exposing that > > > > > > > > means that those kind of messages could not be used > > > > > > > > with pub-sub or radio-dish, as they can't have multiple > > > > > > > > references without copying them, which means there > > > > > > > > would be a semantic difference between the different > > > > > > > > message initialisation APIs, unlike now when the > > > > > > > > difference is only in who owns the buffer. It would > > > > > > > > make the API quite messy in my opinion, and be quite > > > > > > > > confusing as pub/sub is probably the most well known > > > > > > > > pattern. > > > > > > > > > > > > > > > > On Thu, 2019-07-04 at 23:20 +0200, Francesco wrote: > > > > > > > > > Hi Luca, > > > > > > > > > thanks for the details. Indeed I understand why the > > > > > > > > > "content_t" needs > > > > > > > > > to be allocated dynamically: it's just like the > > > > > > > > > control block used by > > > > > > > > > STL's std::shared_ptr<>. > > > > > > > > > > > > > > > > > > And you're right: I'm not sure how much gain there is > > > > > > > > > in removing 100% > > > > > > > > > of malloc operations from my TX path... still I would > > > > > > > > > be curious to > > > > > > > > > find it out but right now it seems I need to patch > > > > > > > > > ZMQ source code to > > > > > > > > > achieve that. > > > > > > > > > > > > > > > > > > Anyway I wonder if it could be possible to expose in > > > > > > > > > the public API a > > > > > > > > > method like "zmq::msg_t::init_external_storage()" > > > > > > > > > that, AFAICS, allows > > > > > > > > > to create a non-shared zero-copy long message... it > > > > > > > > > appears to be used > > > > > > > > > only by v2 decoder internally right now... > > > > > > > > > Is there a specific reason why that's not accessible > > > > > > > > > from the public > > > > > > > > > API? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Francesco > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Il giorno gio 4 lug 2019 alle ore 20:25 Luca Boccassi > > > > > > > > > < > > > > > > > > > [email protected]> ha scritto: > > > > > > > > > > Another reason for that small struct to be on the > > > > > > > > > > heap is so that it > > > > > > > > > > can be shared among all the copies of the message > > > > > > > > > > (eg: a pub socket > > > > > > > > > > has N copies of the message on the stack, one for > > > > > > > > > > each subscriber). > > > > > > > > > > The struct has an atomic counter in it, so that > > > > > > > > > > when all the copies > > > > > > > > > > of the message on the stack have been closed, the > > > > > > > > > > userspace buffer > > > > > > > > > > deallocation callback can be invoked. If the atomic > > > > > > > > > > counter were on > > > > > > > > > > the stack inlined in the message, this wouldn't > > > > > > > > > > work. > > > > > > > > > > So even if room were to be found, a malloc would > > > > > > > > > > still be needed. > > > > > > > > > > > > > > > > > > > > If you _really_ are worried about it, and testing > > > > > > > > > > shows it makes a > > > > > > > > > > difference, then one option could be to pre- > > > > > > > > > > allocate a set of these > > > > > > > > > > metadata structures at startup, and just assign > > > > > > > > > > them when the > > > > > > > > > > message is created. It's possible, but increases > > > > > > > > > > complexity quite a > > > > > > > > > > bit, so it needs to be worth it. > > > > > > > > > > > > > > > > > > > > On Thu, 2019-07-04 at 17:42 +0100, Luca Boccassi > > > > > > > > > > wrote: > > > > > > > > > > > The second malloc cannot be avoided, but it's > > > > > > > > > > > tiny and fixed in > > > > > > > > > > size > > > > > > > > > > > at > > > > > > > > > > > compile time, so the compiler and glibc will be > > > > > > > > > > > able to optimize > > > > > > > > > > it > > > > > > > > > > > to > > > > > > > > > > > death. > > > > > > > > > > > > > > > > > > > > > > The reason for that is that there's not enough > > > > > > > > > > > room in the 64 > > > > > > > > > > bytes > > > > > > > > > > > to > > > > > > > > > > > store that structure, and increasing the message > > > > > > > > > > > allocation on > > > > > > > > > > the > > > > > > > > > > > stack past 64 bytes means it will no longer fit > > > > > > > > > > > in a single cache > > > > > > > > > > > line, which will incur in a performance penalty > > > > > > > > > > > far worse than the > > > > > > > > > > small > > > > > > > > > > > malloc (I tested this some time ago). That is of > > > > > > > > > > > course unless > > > > > > > > > > you > > > > > > > > > > > are > > > > > > > > > > > running on s390 or a POWER with 256 bytes > > > > > > > > > > > cacheline, but given > > > > > > > > > > it's > > > > > > > > > > > part of the ABI it would be a bit of a mess for > > > > > > > > > > > the benefit of > > > > > > > > > > very > > > > > > > > > > > few > > > > > > > > > > > users if any. > > > > > > > > > > > > > > > > > > > > > > So I'd recommend to just go with the second plan, > > > > > > > > > > > and compare > > > > > > > > > > what > > > > > > > > > > > the > > > > > > > > > > > result is when passing a deallocation function vs > > > > > > > > > > > not passing it > > > > > > > > > > (yes > > > > > > > > > > > it will leak the memory but it's just for the > > > > > > > > > > > test). My bet is > > > > > > > > > > that > > > > > > > > > > > the > > > > > > > > > > > difference will not be that large. > > > > > > > > > > > > > > > > > > > > > > On Thu, 2019-07-04 at 16:30 +0200, Francesco > > > > > > > > > > > wrote: > > > > > > > > > > > > Hi Stephan, Hi Luca, > > > > > > > > > > > > > > > > > > > > > > > > thanks for your hints. However I inspected > > > > > > > > > > > > > > > > > > > > > > https://github.com/dasys-lab/capnzero/blob/master/capnzero/src/Publi > > > > > > > > > > sher.cpp > > > > > > > > > > > > and I don't think it's saving from malloc()... > > > > > > > > > > > > see my point > > > > > > > > > > 2) > > > > > > > > > > > > below: > > > > > > > > > > > > > > > > > > > > > > > > Indeed I realized that probably current ZMQ API > > > > > > > > > > > > does not allow > > > > > > > > > > me > > > > > > > > > > > > to > > > > > > > > > > > > achieve the 100% of what I intended to do. > > > > > > > > > > > > Let me rephrase my target: my target is to be > > > > > > > > > > > > able to > > > > > > > > > > > > - memory pool creation: do a large memory > > > > > > > > > > > > allocation of, say, > > > > > > > > > > 1M > > > > > > > > > > > > zmq_msg_t only at the start of my program; > > > > > > > > > > > > let's say I create > > > > > > > > > > all > > > > > > > > > > > > these zmq_msg_t of a size of 2k bytes each > > > > > > > > > > > > (let's assume this > > > > > > > > > > is > > > > > > > > > > > > the > > > > > > > > > > > > max size of message possible in my app) > > > > > > > > > > > > - during application lifetime: call > > > > > > > > > > > > zmq_msg_send() at anytime > > > > > > > > > > > > always avoiding malloc() operations (just > > > > > > > > > > > > picking the first > > > > > > > > > > > > available unused entry of zmq_msg_t from the > > > > > > > > > > > > memory pool). > > > > > > > > > > > > > > > > > > > > > > > > Initially I thought that was possible but I > > > > > > > > > > > > think I have > > > > > > > > > > identified > > > > > > > > > > > > 2 > > > > > > > > > > > > blocking issues: > > > > > > > > > > > > 1) If I try to recycle zmq_msg_t directly: in > > > > > > > > > > > > this case I will > > > > > > > > > > fail > > > > > > > > > > > > because I cannot really change only the "size" > > > > > > > > > > > > member of a > > > > > > > > > > > > zmq_msg_t without reallocating it... so that > > > > > > > > > > > > I'm forced (in my > > > > > > > > > > > > example) > > > > > > > > > > to > > > > > > > > > > > > always send 2k bytes out (!!) > > > > > > > > > > > > 2) if I do create only a memory pool of buffers > > > > > > > > > > > > of 2k bytes and > > > > > > > > > > > > then wrap the first available buffer inside a > > > > > > > > > > > > zmq_msg_t > > > > > > > > > > > > (allocated > > > > > > > > > > on > > > > > > > > > > > > the > > > > > > > > > > > > stack, not in the heap): in this case I need to > > > > > > > > > > > > know when the > > > > > > > > > > > > internals of ZMQ have completed using the > > > > > > > > > > > > zmq_msg_t and thus > > > > > > > > > > when I > > > > > > > > > > > > can mark that buffer as available again in my > > > > > > > > > > > > memory pool. > > > > > > > > > > However > > > > > > > > > > > > I > > > > > > > > > > > > see that zmq_msg_init_data() ZMQ code contains: > > > > > > > > > > > > > > > > > > > > > > > > // Initialize constant message if there's > > > > > > > > > > > > no need to > > > > > > > > > > > > deallocate > > > > > > > > > > > > if (ffn_ == NULL) { > > > > > > > > > > > > ... > > > > > > > > > > > > _u.cmsg.data = data_; > > > > > > > > > > > > _u.cmsg.size = size_; > > > > > > > > > > > > ... > > > > > > > > > > > > } else { > > > > > > > > > > > > ... > > > > > > > > > > > > _u.lmsg.content = > > > > > > > > > > > > static_cast<content_t *> (malloc > > > > > > > > > > > > (sizeof > > > > > > > > > > (content_t))); > > > > > > > > > > > > ... > > > > > > > > > > > > _u.lmsg.content->data = data_; > > > > > > > > > > > > _u.lmsg.content->size = size_; > > > > > > > > > > > > _u.lmsg.content->ffn = ffn_; > > > > > > > > > > > > _u.lmsg.content->hint = hint_; > > > > > > > > > > > > new (&_u.lmsg.content->refcnt) > > > > > > > > > > > > zmq::atomic_counter_t > > > > > > > > > > (); > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > So that I skip malloc() operation only if I > > > > > > > > > > > > pass ffn_ == NULL. > > > > > > > > > > The > > > > > > > > > > > > problem is that if I pass ffn_ == NULL, then I > > > > > > > > > > > > have no way to > > > > > > > > > > know > > > > > > > > > > > > when the internals of ZMQ have completed using > > > > > > > > > > > > the zmq_msg_t... > > > > > > > > > > > > > > > > > > > > > > > > Any way to workaround either issue 1) or issue > > > > > > > > > > > > 2) ? > > > > > > > > > > > > > > > > > > > > > > > > I understand that the malloc is just of > > > > > > > > > > > > size(content_t)~= > > > > > > > > > > 40B... > > > > > > > > > > > > but > > > > > > > > > > > > still I'd like to avoid it... > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > Francesco > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Il giorno gio 4 lug 2019 alle ore 14:58 Stephan > > > > > > > > > > > > Opfer < > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > ha scritto: > > > > > > > > > > > > > On 04.07.19 14:29, Luca Boccassi wrote: > > > > > > > > > > > > > > How users make use of these primitives is > > > > > > > > > > > > > > up to them > > > > > > > > > > though, I > > > > > > > > > > > > > don't > > > > > > > > > > > > > > think anything special was shared before, > > > > > > > > > > > > > > as far as I > > > > > > > > > > remember. > > > > > > > > > > > > > Some example can be found here: > > > > > > > > > > > > > > > > > > > > > > > https://github.com/dasys-lab/capnzero/tree/master/capnzero/src > > > > > > > > > > > > > The classes Publisher and Subscriber should > > > > > > > > > > > > > replace the > > > > > > > > > > publisher > > > > > > > > > > > > > and > > > > > > > > > > > > > subscriber in a former Robot-Operating- > > > > > > > > > > > > > System-based System. I > > > > > > > > > > > > > hope that the subscriber is actually using > > > > > > > > > > > > > the method Luca is > > > > > > > > > > > > > talking > > > > > > > > > > about > > > > > > > > > > > > > on the > > > > > > > > > > > > > receiving side. > > > > > > > > > > > > > > > > > > > > > > > > > > The message data here is a Cap'n Proto > > > > > > > > > > > > > container that we > > > > > > > > > > > > > "simply" > > > > > > > > > > > > > serialize and send via ZeroMQ -> therefore > > > > > > > > > > > > > the name Cap'nZero > > > > > > > > > > ;-) > > > > > > > > > > > > > _____________________________________________ > > > > > > > > > > > > > __ > > > > > > > > > > > > > zeromq-dev mailing list > > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > zeromq-dev mailing list > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > > > > > > > > > > > -- > > > > > > > > Kind regards, > > > > > > > > Luca Boccassi > > > > > > > > _______________________________________________ > > > > > > > > zeromq-dev mailing list > > > > > > > > [email protected] > > > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > _______________________________________________ > > > > > > zeromq-dev mailing list > > > > > > [email protected] > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > > > _______________________________________________ > > > > > > > > > > zeromq-dev mailing list > > > > > > > > > > [email protected] > > > > > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > > > > > > _______________________________________________ > > > > > > > > zeromq-dev mailing list > > > > > > > > [email protected] > > > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > > > > _______________________________________________ > > > > > > zeromq-dev mailing list > > > > > > [email protected] > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > > > > _______________________________________________ > > > > zeromq-dev mailing list > > > > [email protected] > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________zeromq-dev mailing > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev -- Kind regards, Luca Boccassi
signature.asc
Description: This is a digitally signed message part
_______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
