You may be able to get more space than what you currently have, but my
understanding is the XP in a default configuration can only address
2.5 GB. You also need to consider that you will need open memory in
order to do anything useful. I am not aware of limitations imposed by
rbind per se. Tables (if you are using the R termonology) are actually
special forms of arrays.
I just ran this test and it used about half of my 10 GB according to
my systemmonitor:
> mtx <- matrix(1:(2500*150), ncol=150)
> for (i in 1:10) { mtx <- rbind(mtx,mtx); print( 2^i) }
[1] 2
[1] 4
[1] 8
[1] 16
[1] 32
[1] 64
[1] 128
[1] 256
[1] 512
[1] 1024
Running it again only 8 times and looking at the fraction of occupied
memory with Apple's Activity Monitor makes me think you could get 256
of such tables into your space without too much difficulty, but the
number could easily be twice that since my pointer overhead in a 64-
bit system will be greater than yours. You will probably even have
room to do something useful. The option of leaving the data in a
database should not be overlooked if you need more space. sqldf is an
available package:
http://code.google.com/p/sqldf/
Other informative functions include object.size() and gc(). After
doubling that 8-fold copy of mtx to get a nine-fold doubling, I see:
> str(mtx)
int [1:1280000, 1:150] 1 2 3 4 5 6 7 8 9 10 ...
> object.size(mtx)
[1] 768000200
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 114618 6.2 350000 18.7 350000 18.7
Vcells 96121989 733.4 309848649 2364.0 576124419 4395.5
Good luck;
David
On Jun 20, 2009, at 8:49 AM, gug wrote:
Thanks David - you're right, "PC" is not very informative. I am
using XP
Home with 4GB, though I don't think XP accesses more than 3GB.
From following through the FAQ's and memory functions (e.g.
"memory.limit(size = NA)") it looks like R is getting about 1535Mb
at the
moment.
David Winsemius wrote:
My question is: What is the maximum number of tables that can be
combined
with rbind?
Is it driven by the number of rows in the tables, by a constraint
with
the
syntax of rbind itself, by my PC's memory,
Yes, and the OS. "PC" is not a useful designation for an OS.
or by some other constraint? At this point the data files I am
reading
in are approx 2,500 rows x 150
columns.
That should not be any problem. On an Intel-Mac w/ 8 GB there is no
problem with dataframes that are 500 times that size.
If this is on a Windows box, there is an RW-FAQ on the topic.
David Winsemius, MD
Heritage Laboratories
West Hartford, CT
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
