First, we strongly recommend 64-bit R. Otherwise, you may not be able
to scale up as far as you would like.
Second, as I think you realize, with big objects you may have to do
things in chunks. I generally recommend working a column at a time
rather than in blocks of rows if possible (better per
I recently learned about the bigmemory and foreach packages and am trying
to use them to help me create a very large matrix. Without those
packages, I can create the type of matrix that I want with 10 columns and
5e6 rows. I would like to be able to scale up to 5e9 rows, or more, if
possible.
I have a piece of code as the one at the bottom, unfortunately since it
involves time series from a db it's not easy to give to mailing list a working
script.
It becomes very slow after few hundred iterations over variable sp (must
process several thousands).
The Rprof() indicates that the probl
3 matches
Mail list logo