On 14/07/2010 5:23 PM, paul s wrote:
hi -
i just started using R as i am trying to figure out how perform a linear
regression on a huge matrix.
i am sure this topic has passed through the email list before but could
not find anything in the archives.
i have a matrix that is 2,000,000 x 170,000 the values right now are
arbitray.
i try to allocate this on a x86_64 machine with 16G of ram and i get the
following:
> x <- matrix(0,2000000,170000);
Error in matrix(0, 2e+06, 170000) : too many elements specified
>
is R capable of handling data of this size? am i doing it wrong?
It is capable of handling large data, but not that large in a single
matrix. The limit on the number of entries in any vector (and matrices
are stored as vectors) is about 2^31 ~ 2 billion. Your matrix needs
about 340 billion entries, so it's too big. (It won't all fit in
memory, either: you've only got space for 2 billion numeric values in
your 16G of RAM, and you also need space for the OS, etc. But the OS
can use disk space as virtual memory, so you might be able to get that
much memory, it would just be very, very slow.)
You need to break up the work into smaller pieces.
Duncan Murdoch
cheers
paul
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
