I'm more and more convinced this is a heap-fragmentation problem; mainly
because I can't find any actual memory leaks when dnsmasq actions a SIGHUP.
I've just pushed 2.93test4, which addresses this for the DNS subsystem,
(ie names (re)read from /etc/hosts and friends) On sending SIGHUP the
same memory is re-used, so unless the files get significantly larger, or
you're very unlucky, no calls to free() or malloc() are made.
There's another source of heap churn that happens on SIGHUP, which is
the read-reading of dhcp-hostfile (and dhcp-optsfile). test4 doesn't do
anything about this, but the same sort of approach would work. I don't
think that's relevant on a standard OpenWRT install: the names from
odhcp6 leases are read from a hostsfile format file. The logs Y.Y.
posted on the openWRT github indicate that it's not relevant on the
system he's testing.
Also, in case you missed it: test2 fixed a bug where a static buffer got
free and re-allocated every time, rather than just when it needed to
expand. That alone would churn a lot of heap.
Y.Y. I'm very interested to know if test4 improves things for you.
The leak on OpenSense seems to be at a larger magnitude and SIGHUPs
there are infrequent, so I don't hold out much hope for that, we really
need more test results to make progress there.
Cheers,
Simon.
On 02.02.2026 08:20, Y.Y. wrote:
Hi Simon,
*
OpenWrt defaults to musl and this cannot be changed. It’s the
standard environment for millions of these devices.
*
The script is an "accelerator"， Sub-second SIGHUPs aren't as rare
as my case, Each time an IPv6 address expires, when OpenWrt
maintains the mapping between local domain names and IP addresses,
it triggers SIGHUP multiple times within one second. I have
previously posted the actual logs about this on GitHub. —in my case,
the exhaustion takes weeks to show up, which is why this bug has
haunted me for years. The script is just to speed up the observation.
*
Even at 100 SIGHUPs per second, the memory rises slowly—it’s just a
matter of duration. However, using a larger config file will
significantly speed up this process.
Here is a much more slow script with at most 100 SIGHUPs per second
to expose this behavior
while :;do top -bn1 | grep [d]nsmasq |grep -v ujail | awk '{print
$3 "| Memory is about: " $5*1024 " B"}';sleep 1;done
while :;do PID=$(pidof dnsmasq);for i in $(seq 1 100); do kill -
SIGHUP $PID;sleep 1; done;done
*
If you throttle the VM's RAM and CPU, the trend becomes obvious much
sooner. You don't need half a million HUPs to see it if the system
resources are constrained.
I have an x86_64 VM image ready with the latest OpenWrt snapshot if
you want to test it in a this environment.
Regards.
Simon Kelley <[email protected] <mailto:[email protected]>>
于2026年1月31日周六 00:53写道：
I just sent SIGHUP twice in succession to the dnsmasq process in my
OpenWRT router, with the new malloc-logging feature enabled.
HUP frees a load of configuration and the re-reads it and I correlated
all the memory freed by the second HUP with what was allocated in the
first HUP.
It's perfect. Every block is freed.
This is a fairly old installation, so old libraries, etc, but the very
latest dnsmasq code.
The configuration it's re-reading is pretty small.
I then tried your technique of hitting dnsmasq hard with many HUPs.
I had to go up to half a million to see much effect, but I guess
most of
those were dropped since they will have arrived before the previous one
was cleared.
In any case I could see a reproducible rise of a few percent in the VSZ
of the process each time.
What's clear is that the configuration is stored in a _lot_ of small
allocations, so re-reading a substantial configuration will free a lot
of small blocks and then malloc a lot of small blocks.
A quick Google produces some complaints about the fragmentation
performance of musl, which may be significant.
Is your installation using musl as the C library, and is it possible to
build dnsmasq against, say glibc to test?
Nearly all of the memory management on dnsmasq that gets hit by
answering DNS or DHCP requests avoid hammering the malloc system by
building pools of free data structures that get re-cycled as needed.
Once the pools have grown to equilibrium size, even a very busy server
hardly uses the heap. I guess the configuration code to use the same
policy, but it's a big re-write, and re-reading configuration on a
sub-second timescale is an unlikely use-case.
Cheers,
Simon.
On 30.01.2026 01:59, Y.Y. wrote:
> *Hi Simon,*
>
> I think Your suspicion about the configuration re-reading logic
is spot
> on. I have documented the reproduction and some analysis here:
https://
> github.com/openwrt/openwrt/issues/21729#issuecomment-3815442601
<http://github.com/openwrt/openwrt/
issues/21729#issuecomment-3815442601>
> <https://github.com/openwrt/openwrt/
issues/21729#issuecomment-3815442601 <https://github.com/openwrt/
openwrt/issues/21729#issuecomment-3815442601>>
>
> To assist with the debugging, I can provide an OpenWrt *Linux
x86_64 VM
> image* that consistently reproduces this memory leak. Please let
me know
> if this would be helpful for your investigation.
>
>
> _______________________________________________
> Dnsmasq-discuss mailing list
> [email protected] <mailto:Dnsmasq-
[email protected]>
> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-
discuss <https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/
dnsmasq-discuss>
_______________________________________________
Dnsmasq-discuss mailing list
[email protected] <mailto:Dnsmasq-
[email protected]>
https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-
discuss <https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/
dnsmasq-discuss>
_______________________________________________
Dnsmasq-discuss mailing list
[email protected]
https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
_______________________________________________
Dnsmasq-discuss mailing list
[email protected]
https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
