Bug#827953: maxima: Uses too much memory to build

Santiago Vila Wed, 14 Sep 2016 04:54:53 -0700

Hello Camm.

Nice to hear from you again.

On Tue, 13 Sep 2016, Camm Maguire wrote:

> It appears that all is working as intended, that gc is triggered and
> keeping the active heap to about 2.8Gb (or 3.9Gb with relocatable copy
> area) of a detected max of 5.3Gb (see core pages, and physical memory).
> At the end of this experiment, the rss of this process as viewed from a
> separate shell should confirm these figures.  So I do not think gcl is
> expanding into swap here.

Ok, even if it's working as intended, maybe the intended way is still
not the optimal way :-), and maybe we might still consider to change the
intended way. I'll elaborate on that below.

> As for the /proc/meminfo, all I can say is that the build employs many
> other programs as well.  Might be interesting to correlate this figure
> with the various stages of the build.  In other words, how much just to
> run the debian/rules build target, etc.

That's probably the problem.

The garbage collection procedure may be very well designed not to take
all available memory, but this good care it takes ends up being not enough
when we consider all the other processes involved in the build of the
package, i.e. the build process as a whole.

The current procedure is a little bit "egoistical", so to speak, as it
takes almost all the available memory for himself, not leaving enough memory for
the other processes.

The problem I'd like to see fixed is the amount of memory used by the
Debian package build as a whole, for which the garbage collection thing is
only a part.

If you can fix this problem by fine-tuning GCL, fine, but if not,
I'd like you to consider the problem as a whole, not just the garbage
collection thing.

Maybe this is just a matter of setting

GCL_MEMORY_something=somevalue

somewhere in debian/rules and nothing less. I really don't know.

But I'll better describe what I do in case you want to replicate my
results. It's quite simple indeed.

When my autobuilder starts building a package (just before it starts),
it takes note of the value of Committed_AS in /proc/meminfo.

Then the build starts and it keeps taking notes of Committed_AS at every
second, then substracts the initial value and stores the difference.

At the end I have a text file with the amount of memory that it was
committed during the build. I call this "memory profile".

(My autobuilder does not do anything else other than building a
package at a time, so this is as accurate as it can be with the
limited available resources I have).

I attach the two memory profiles I have for maxima.

I do not remember how much memory these two autobuilders had, because
I often change the configuration from time to time, but I'm almost
sure that I did not have 17 GB as required by "lutecio" (the second
attach). If my records are correct, this was a Linode with only
8 GB of RAM (and maybe a similar amount of swap).

So, to summarize: Would you please try observing Committed_AS in
/proc/meminfo while the Debian maxima package is building to see if
you can reproduce what I report, i.e. that the amount of memory it
takes is usually a lot more than the available RAM?

Thanks a lot.

Bug#827953: maxima: Uses too much memory to build

Reply via email to