On Fri, Nov 22, 2024 at 11:36 AM David Scott <d.sc...@epcc.ed.ac.uk> wrote:
> Hello,
>
> I am using the options mechanism of PETSc to configure my CFD code. I
> have introduced options describing the size of the domain etc. I have
> noticed that this consumes a lot of memory. I have found that the amount
> of memory used scales linearly with the number of MPI processes used.
> This restricts the number of MPI processes that I can use.
>
There are two statements:
1) The memory scales linearly with P
2) This uses a lot of memory
Let's deal with 1) first. This seems to be trivially true. If I want every
process to have
access to a given option value, that option value must be in the memory of
every process.
The only alternative would be to communicate with some process in order to
get values.
Few codes seem to be willing to make this tradeoff, and we do not offer it.
Now 2). Looking at the source, for each option we store a PetscOptionItem,
which I count
as having size 37 bytes (12 pointers/ints and a char). However, there is
data behind every
pointer, like the name, help text, available values (sometimes), I could
see it being as large
as 4K. Suppose it is. If I had 256 options, that would be 1M. Is this a
large amount of memory?
The way I read the SLURM output, 29K was malloced. Is this a large amount
of memory?
I am trying to get an idea of the scale.
Thanks,
Matt
> Is there anything that I can do about this or do I need to configure my
> code in a different way?
>
> I have attached some code extracted from my application which
> demonstrates this along with the output from a running it on 2 MPI
> processes.
>
> Best wishes,
>
> David Scott
> The University of Edinburgh is a charitable body, registered in Scotland,
> with registration number SC005336. Is e buidheann carthannais a th’ ann an
> Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!aDTOdIHWWilf4sShnRrU9KcJ987GlIrJ71v1EcIH4zje2tKZ7EBoEBD2TqNejin_X3-7DKujGeq-pXHvyHqF$
<https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!aDTOdIHWWilf4sShnRrU9KcJ987GlIrJ71v1EcIH4zje2tKZ7EBoEBD2TqNejin_X3-7DKujGeq-pWu0-nb6$
>
