On Sun, 4 Jun 2023, László Böszörményi wrote:
Hi,
On Sat, Jun 3, 2023 at 8:30 PM Bob Friesenhahn
<bfrie...@simple.dallas.tx.us> wrote:
I am definitely able to confirm that memory consumption builds due to
invoking GetImageDepth() via a POSIX thread. The rate that it builds
is image sensitive since some images cause GetImageDepth() to perform
more OpenMP loops.
Unfortunately I can not reproduce. My processor is an Intel K variant
CPU, six cores and twelve threads, 64 Gb of RAM.
GM is 1.3.40 with two security fixes backported, compiled with GCC
v12.2.0. Tried with three PNG images, all memory consumption is static
from the beginning. Do I need some special case of PNG files to
experience this issue?
Using PNG is not important. The nature of the input image is
important. Prepare a test image like
gm convert infile.png -depth 4 testfile.pnm
and then use testfile.pnm as input.
The PNM reader is now also also threaded (in very recent versions).
PNG should work as well to reproduce the bug.
My own testing is under Ubuntu 20.04 using GCC 10.
Do you think it might be a problem with another system component, a
GCC optimization or this is fixed meanwhile? At least I do wonder why
this issue is CPU / machine dependent.
That of course is the question. It seems like some small bit of state
is being added to (presumably) many thread stacks, I see the number of
'anon' mappings increasing, and the "heap" similarly grows and, but is
released before the program exits.
It is possible that doing some OpenMP thing prior to using POSIX
threads (so OpenMP teams are already created) might change the
outcome. I have not tried that yet.
I have attached a version of threadarena.c which loops 1024 times and
then quits.
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
/*
* gcc -o threadarena threadarena.c \
* `GraphicsMagick-config --cppflags --ldflags --libs`
*/
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <limits.h>
#include <string.h>
#include <time.h>
#include <sys/types.h>
#include <pthread.h>
#include <magick/api.h>
static void* readimage(void* arg)
{
ImageInfo *imageInfo;
ExceptionInfo exception;
const char* filename = (const char*) arg;
/* ImageCharacteristics chars; */
unsigned long depth;
imageInfo=CloneImageInfo(0);
GetExceptionInfo(&exception);
(void) strncpy(imageInfo->filename, arg, MaxTextExtent-1 );
Image *image = ReadImage(imageInfo, &exception);
if (image == (Image *) NULL)
{
CatchException(&exception);
perror("ReadImage");
}
depth=GetImageDepth(image,&exception);
if (exception.severity != UndefinedException)
CatchException(&exception);
else
fprintf(stdout,"Depth: %lu\n", depth);
DestroyImage(image);
DestroyImageInfo(imageInfo);
DestroyExceptionInfo( &exception );
return NULL;
}
int main ( int argc, char **argv )
{
unsigned int count = 1024;
const char* file = argv[1];
if ( file == NULL )
{
fprintf(stderr,"usage: %s <imagefile>\n", argv[0]);
return 1;
}
InitializeMagick(NULL);
while (count)
{
pid_t myp = getpid();
char s[PATH_MAX];
#if 1
/* this code path has a thread arena leak */
pthread_t thread_id;
pthread_create(&thread_id, NULL, readimage, argv[1]);
/* pthread_detach(thread_id); also shows the problem */
pthread_join(thread_id, NULL);
#else
/* this code path doesn't have a thread arena leak */
readimage(argv[1]);
#endif
sleep(1);
fprintf(stderr,"%u : %lld\n", count, (long long)time(0));
snprintf(s,sizeof(s),
"ps -o vsz -o rss -o user -o command -p %lld", (long long)myp);
system(s);
snprintf(s,sizeof(s),
"pmap -x %lld | grep anon | wc -l", (long long)myp);
system(s);
count--;
}
DestroyMagick();
return 0;
}
