Hi Kevin, I ran your test code and saw the same issue. I also created the tests with straight libzmq (not czmq) and the issue still occurs.
The memory is not lost (valgrind confirms) but it does grow. Eventually I noticed its proportional to the number of times you call zmq_recv(). Adding code to force heap cleanup and print the stats shows that every 100 times zmq_recv is called, the queue is purged. malloc_trim(0); malloc_stats(); Finally I found inbound_poll_rate which is used by socket_base to determine how often to check for signals and commands. https://github.com/zeromq/libzmq/blob/b77d7610cd91dbe25096b804c77b27065b7dd1fd/src/config.hpp#L53 Reducing this 1 (as a test) fixed the issue and memory never rose above 4Mb. Is this a real issue? I am not sure. I guess for some edge cases where you receive a truck load of data and don’t process it. But on the other hand why would you do that. Possible fixes I can think of would be a socket_option to modify inbound_poll_rate OR Force process_commands() on a timer if no activity is occurring. Thanks James -----Original Message----- From: zeromq-dev <[email protected]> On Behalf Of Kevin Wang Sent: 20 June 2018 22:12 To: [email protected] Subject: Re: [zeromq-dev] Memory leak on SUB set with CONFLATE? The memory of the SUB process goes up. If I remove the getline() call and the SUB process keeps up, the RAM stays still for both. Those programs compile with gcc pretty easily, if you feel like trying it out. Maybe my Ubuntu is just messed up. I assume since its CONFLATE is on, maybe the PUB doesn't queue it, but SUB still does? In that case, I also would assume that replacing the old message with the new one would free the memory held by the old message. On 06/20/2018 01:38 PM, James Harvey wrote: > When you say “the system memory skyrockets” are you talking about the memory > used by the SUB process or PUB process? > > I would expect the memory of the PUB process to increase as it will be > queueing messages. > >> On 20 Jun 2018, at 21:02, Kevin Wang <[email protected]> wrote: >> >> the system memory skyrockets, > _______________________________________________ > zeromq-dev mailing list > [email protected] > https://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
