Thank you all for your help!
I used the function "big.matrix", from a package named "bigmemory", to
allocate a large matrix 50k x 50k
(http://www.inside-r.org/packages/cran/bigmemory/docs/bigmemory).
And I followed the suggestions from Sarah to do the calculations fastly!
Thank you very much!
Best!
Charles
On Tue, Dec 4, 2012 at 9:33 PM, Sarah Goslee <sarah.gos...@gmail.com> wrote:
> I don't think there's any reason for the calculation you're doing that
> you must have the whole matrix in memory, is there?
>
> Unless there's something more than what you've shown us, you're just
> taking the sum of elementwise operations. You can read the matrix in
> in manageable chunks, take the sum of that chunk and save the single
> value. Repeat, then add them all up at the end.
>
> Sarah
>
> On Tue, Dec 4, 2012 at 3:14 PM, Charles Novaes de Santana
> <charles.sant...@gmail.com> wrote:
>> On Tue, Dec 4, 2012 at 8:43 PM, Peter Langfelder
>> <peter.langfel...@gmail.com> wrote:
>>> On Tue, Dec 4, 2012 at 11:27 AM, Charles Novaes de Santana
>>> <charles.sant...@gmail.com> wrote:
>>>> Dear Michael,
>>>>
>>>> Thank you for your answer.
>>>>
>>>> I have 2 matrices. Each position of the matrices is a weight. And I
>>>> need to calculate the following sum of differences:
>>>>
>>>> Considering:
>>>> mat1 and mat2 - two matrices (each of them 48000 x 48000).
>>>> d1 and d2 - two constant values.
>>>>
>>>> sum<-0;
>>>> for(i in 1:nrows1){
>>>> for(j in 1:nrows2){
>>>> sum<-sum+ ( ( (mat1(i,j)/d1) -
>>>> (mat2(i,j)/d2) )^2 )
>>>> }
>>>> }
>>>> }
>>>>
>>>> I was wondering if there is a better way to do this sum.
>>>
>>> sum( (mat1/d1-mat2/d2)^2)
>>>
>>> Correct me if I'm wrong though - aren't matrices of 48x times 48k
>>> larger than what R can handle at present?
>>>
>>> HTH
>>>
>>> Peter
>>
>> hmmm I didn't know that the limitation of R was below this value. I
>> found this error message:
>>
>> "Error in matrix(0, 48000, 48000) : too many elements specified"
>>
>> but I thought it was a machine limitation (and I was asking for access
>> to a better machine in my labs...). Thanks for clarifying it.
>>
>> Well, when Sarah gave me the answer for my problem, I got a new one :)
>> Thank you, Sarah and Peter.
>>
>> So, is there any other way to "trick R" and allocate such large matrices?
>>
>> Best,
>>
>> Charles
>> --
>> Um axé! :)
>>
>
> --
> Sarah Goslee
> http://www.functionaldiversity.org
--
Um axé! :)
--
Charles Novaes de Santana
http://www.imedea.uib-csic.es/~charles
PhD student - Global Change
Laboratorio Internacional de Cambio Global
Department of Global Change Research
Instituto Mediterráneo de Estudios Avanzados(CSIC/UIB)
Calle Miquel Marques 21, 07190
Esporles - Islas Baleares - España
Office phone - +34 971 610 896
Cell phone - +34 660 207 940
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
