Package: graphicsmagick
Version: 1.4+really1.3.36+hg16481-2+deb11u1
Severity: normal
Tags: upstream
The GraphicsMagick API has a thread arena and more general memory leak
somewhere in the GetImageDepth() API call, presumably a side-effect of
the OpenMP usage.
First, you need a server class system... a lot of beefy cores (i.e. I
can't reproduce it on my quad-core laptop), and possibly a lot of RAM
(128G on this one):
root@host:~# grep 'model name' /proc/cpuinfo |tail -1
model name : Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
root@host:~# grep processor /proc/cpuinfo |tail -1
processor : 39
Then you need this test program:
root@host:~# cat threadarena.c
/*
* gcc -o threadarena threadarena.c \
* `GraphicsMagick-config --cppflags --ldflags --libs`
*/
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <limits.h>
#include <string.h>
#include <time.h>
#include <sys/types.h>
#include <pthread.h>
#include <magick/api.h>
static void* readimage(void* arg) {
ImageInfo *imageInfo;
ExceptionInfo exception;
const char* filename = (const char*) arg;
ImageCharacteristics chars;
imageInfo=CloneImageInfo(0);
GetExceptionInfo(&exception);
(void) strncpy(imageInfo->filename, arg, MaxTextExtent-1 );
Image *image = ReadImage(imageInfo, &exception);
if (image == (Image *) NULL)
{
CatchException(&exception);
perror("ReadImage");
}
GetImageDepth(image,&image->exception);
DestroyImage(image);
DestroyImageInfo(imageInfo);
DestroyExceptionInfo( &exception );
return NULL;
}
int main ( int argc, char **argv )
{
const char* file = argv[1];
if( file == NULL ) {
fprintf(stderr,"usage: %s <imagefile>\n", argv[0]);
return 1;
}
InitializeMagick(NULL);
while(1) {
pid_t myp = getpid();
char s[PATH_MAX];
#if 1
/* this code path has a thread arena leak */
pthread_t thread_id;
pthread_create(&thread_id, NULL, readimage, argv[1]);
/* pthread_detach(thread_id); also shows the problem */
pthread_join(thread_id, NULL);
#else
/* this code path doesn't have a thread arena leak */
readimage(argv[1]);
#endif
sleep(1);
fprintf(stderr,"%lld\n", (long long)time(0));
snprintf(s,sizeof(s),
"ps -o vsz -o rss -o user -o command -p %lld", (long
long)myp);
system(s);
snprintf(s,sizeof(s),
"pmap -x %lld | grep 65404 | wc -l", (long long)myp);
system(s);
}
DestroyMagick();
return 0;
}
Compile and run the test program:
root@host:~# gcc -o threadarena threadarena.c `GraphicsMagick-config --cppflags
--ldflags --libs`
root@host:~# ./threadarena Zoom140.png
1685713732
VSZ RSS USER COMMAND
2443096 8248 root ./threadarena Zoom140.png
35
1685713734
VSZ RSS USER COMMAND
4802392 9816 root ./threadarena Zoom140.png
70
1685713735
VSZ RSS USER COMMAND
7161688 11352 root ./threadarena Zoom140.png
105
1685713736
VSZ RSS USER COMMAND
9520984 12920 root ./threadarena Zoom140.png
140
1685713737
VSZ RSS USER COMMAND
11880280 14456 root ./threadarena Zoom140.png
175
1685713738
VSZ RSS USER COMMAND
14239576 16040 root ./threadarena Zoom140.png
210
1685713739
VSZ RSS USER COMMAND
16598872 17608 root ./threadarena Zoom140.png
245
1685713740
VSZ RSS USER COMMAND
18958168 19128 root ./threadarena Zoom140.png
280
1685713741
VSZ RSS USER COMMAND
20989784 20660 root ./threadarena Zoom140.png
310
1685713742
VSZ RSS USER COMMAND
20989784 22100 root ./threadarena Zoom140.png
309
1685713743
VSZ RSS USER COMMAND
20989784 23492 root ./threadarena Zoom140.png
308
1685713744
VSZ RSS USER COMMAND
20989784 24900 root ./threadarena Zoom140.png
307
1685713745
VSZ RSS USER COMMAND
20989784 26292 root ./threadarena Zoom140.png
306
As you can see, the number of thread arenas (65404 k mapped sections)
according to pmap grows dramatically with each invocation of
GetImageDepth(), and the virtual memory usage blows up to ~20G before
stabilizing. RSS also increases, albeit less dramatically, but eventually
the system does start to hit an OOM condition.
My understanding of malloc thread arenas is that new threads are supposed
to reuse the arenas, and only attempt to create new ones when an existing
arena is locked, so I assume that's what's happening here. And I also assume
there's some missing housekeeping that's also leaking RSS memory.
Some fun facts about the bug:
1. so far, only GetImageDepth() triggers the error. Reading pixel data and
other transformations seems fine.
2. it only happens when the API is called from within a thread (hence the
pthread_start()). If you call it from the main process, no problem.
3. the problem can be mitigated by mallopt(M_ARENA_MAX, n); for some fixed
n (currently running with 8), although I haven't really quantified what that
does to
performance. It seems to stop the RSS leak.
4. I found suggestions that a periodic malloc_trim(0) might mitigate this,
but it doesn't seem to have any impact.
5. obviously, this will only be an issue for long-running GraphicsMagick
processes, such as FastCGI type of applications.
-- System Information:
Debian Release: 11.7
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.0-23-amd64 (SMP w/40 CPU threads)
Locale: LANG=, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages graphicsmagick depends on:
ii libc6 2.31-13+deb11u6
ii libgraphicsmagick-q16-3 1.4+really1.3.36+hg16481-2+deb11u1
graphicsmagick recommends no packages.
Versions of packages graphicsmagick suggests:
pn graphicsmagick-dbg <none>
-- no debconf information
