On 20/06/2013 15:45, Jonathan Greenberg wrote:
Folks:
I apologize for the cross-posting between r-help and r-sig-hpc, but I
figured this question was relevant to both lists. I'm writing a
function to be applied to an input dataset that will be broken up into
chunks for memory management reasons and for parallel execution. I am
trying to determine, for a given function, what the *maximum* memory
usage during its execution is (which may not be the beginning or the
end of the function, but somewhere in the middle), so I can "plan" for
the chunk size (e.g. have a table of chunk size vs. max memory usage).
Is there a trick for determining this?
Note that your subject line and the body of your message are different
questions.
You cannot determine the memory usage of any part of R, in particular
not of a function's execution. Objects are shared, garbage collection
happens asynchronously ....
However, gc() is a good start. Call gc(reset = TRUE) before and gc()
after your task, and you will see the maximum extra memory used by R in
the interim. (This does not include memory malloced by compiled code,
which is much harder to measure as it gets re-used.)
Note that calls to gc() do affect the usage, and the usage also depends
on what had already been done in the session (as the trigger values
adapt to usage).
--j
--
Jonathan A. Greenberg, PhD
Assistant Professor
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
607 South Mathews Avenue, MC 150
Urbana, IL 61801
Phone: 217-300-1924
http://www.geog.illinois.edu/~jgrn/
AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
