On Mon, 15 Dec 2008, tsunhin wong wrote:
Dear R Users,
I was running some data analysis scripts and ran into this error:
Error: cannot allocate vector of size 27.6 Mb
Doing a "memory.size(max=TRUE)" will give me:
[1] 1506.812
The current situation is:
I'm working on a Windows Vista 32bit laptop with 4GB RAM (effectively
3GB I assume...)
Why do you assume? Reading the rw-FAQ should make clear that a 2GB
address limit is a lot more plausible given the data you presented.
I have a data file of 450Mb loaded into R and have around 1500
data.frames floating in the global space as my data source.
The way I run this analysis:
I call a patch processing & procedure script
'batch'?
it retrieves 4 lists of info (each around 400x100) from an index data.frame,
and then it calls another script to retrieve info from the corresponding
data.frames on the 4 lists in the global space
through calling another script, about 1000x3 will be retrieved by another script
the 1000x3 will be passed to a third script expanded to 20001x3, and only
20001x1 will be used
20001x1 will accumulate into a matrix of up to 20001x1500 (number of data
frames / trials), say I have to divide the trials into 2 groups and do a
comparison, then that's processing of 2 matrices of size 20001x750
But the allocation error stopped the script after script has processed
around 280 data frames, i.e. made the first matrix up to 20001x280...
I know running the analysis should possibly be achieved by
restructuring my script a little bit, but I have no idea where to
start with to try...
Run the script on each 100 data frames in turn. You can even use R to set
up scripts for Rscript to run.
Also, I have no idea about Garbage Collection ability or memory
recycle / reuse ability in R and I think some memory may have been
lost in the middle of the process, and it may be possible to put them
back to the system for R to make use of...
Well, do read the help page ?"Memory-limits' as well as the appropriate
FAQ. (And also the parts of the posting guide about doing your homework
before posting. Since this list is a public record, we do need to correct
false claims for the sake of future enquirers doing their homework before
posting.)
Please advise me to let me to find out the most efficient way of
eliminating the error...
Thanks so much!
Regards,
John
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
