Hi Michel, thanks for the suggestion. I will definitely gie it a try. I tried tcmalloc and jemalloc once, but the improvement was not so big. If I understood tcmalloc correctly they use a per-thread cache for allocations. I didn't think of gc.
In my system I have a component which basically receives a data stream (260MB/s, one data frame is 1129 bytes) from an external source, extracts a stream identifier and publishes the data again on a PUB socket with the stream id as a topic. Each frame of 1129 is published as a message part. It also does some filtering and integrate control sockets for RPC, but this is negligible performance-wise. When I profile this I see malloc/free as one of the hot spots. I can also improve performance by increasing the recv buffer in ZeroMQ, but malloc still stays of course. The application is well within the requirements and has enough margin, but I am interested in tuning options. Cheers, Jens -- Jens Auer | CGI | Software-Engineer CGI (Germany) GmbH & Co. KG Rheinstraße 95 | 64295 Darmstadt | Germany T: +49 6151 36860 154 [email protected]<mailto:[email protected]> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>. CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail. ________________________________ Von: zeromq-dev [[email protected]]" im Auftrag von "Michel Pelletier [[email protected]] Gesendet: Dienstag, 29. November 2016 20:16 An: ZeroMQ development list Betreff: Re: [zeromq-dev] On hooking memory allocations On Tue, Nov 29, 2016 at 12:17 AM, Auer, Jens <[email protected]<mailto:[email protected]>> wrote: Hi, A memory-pool with fixed blocks for incoming messages is exactly what I have in mind. E.g. in my application, I receive messages of ~1kb at a rate of 10000/s. Right now with ZeroMQ 4.1, this is done by receiving from the socket into a fixed 8k buffer and then allocating a message with a buffer of 1kb where the data is copied. You might want to consider using an allocation management library like the Boehm garbage collector: http://www.hboehm.info/gc/ It's very good at pooling and is very efficient. In addition it can be used as a leak detection tool and can be used as a drop in replacement for new/malloc/realloc without code changes. -Michel
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
