Thank you both for your advice. I ended up implementing both solutions and
testing them on a real dataset of 10,000 rows and 50 inds. The results are
very, very interesting.
For some context, the original two approaches, nested lapply and nested for
loops, performed at 1.501529
and 1.458963 mi
On Fri, Oct 8, 2010 at 12:47 PM, Gabor Grothendieck
wrote:
> On Fri, Oct 8, 2010 at 11:35 AM, epowell wrote:
>>
>> My data looks like this:
>>
>>> data
>> name G_hat_0_0 G_hat_1_0 G_hat_2_0 G_0 G_hat_0_1 G_hat_1_1 G_hat_2_1 G_1
>> 1 rs0 0.488000 0.448625 0.063375 1 0.480875 0.454500 0.0
On Fri, Oct 8, 2010 at 11:35 AM, epowell wrote:
>
> My data looks like this:
>
>> data
> name G_hat_0_0 G_hat_1_0 G_hat_2_0 G_0 G_hat_0_1 G_hat_1_1 G_hat_2_1 G_1
> 1 rs0 0.488000 0.448625 0.063375 1 0.480875 0.454500 0.064625 1
> 2 rs1 0.002375 0.955375 0.042250 1 0.00 0.06
You are loosing a lot of time by repeatedly calculating character
indices with paste() in every iteration. Two options:
-- 1) calculate these once outside the loop and then refer to them by
index
idx.names <- vector(mode="character", length=nind)
for (i in (0:(nind-1))) {idx[i+1] <-# ne
My data looks like this:
> data
name G_hat_0_0 G_hat_1_0 G_hat_2_0 G_0 G_hat_0_1 G_hat_1_1 G_hat_2_1 G_1
1 rs0 0.488000 0.448625 0.063375 1 0.480875 0.454500 0.064625 1
2 rs1 0.002375 0.955375 0.042250 1 0.00 0.062875 0.937125 2
3 rs2 0.050375 0.835875 0.113750
On Tue, Jun 8, 2010 at 11:53 PM, Worik R wrote:
> Given the following snippet
> m.nf.xts <- xts(rep(0, length(index(m.xts))), order.by=index(m.xts))
>
> Does R know to cache the index(m.xts) or is it more efficient to say...
>
> m.i <- index(m.xts)
> m.nf.xts <- xts(rep(0, length(m.i)), orde
Given the following snippet
m.nf.xts <- xts(rep(0, length(index(m.xts))), order.by=index(m.xts))
Does R know to cache the index(m.xts) or is it more efficient to say...
m.i <- index(m.xts)
m.nf.xts <- xts(rep(0, length(m.i)), order.by=index(m.i))
?
cheers
Worik
[[alternative HTML
Gabor, thanks a lot!
I removed everything from the work space but the data frame - and then
DF[is.na(DF)]<-0 has worked!
Thanks a lot!
Dimitri
On Mon, Mar 29, 2010 at 8:45 PM, Gabor Grothendieck
wrote:
> Its going to be pretty hard to do anything useful if you can`t even do
> simple operations li
Its going to be pretty hard to do anything useful if you can`t even do
simple operations like that without overflowing memory but anyways try
this (untested):
write.table(DF, "DF.csv", sep = ",", quote = FALSE)
rm(DF)
DF <- read.csv(pipe("sed s/NA/0/g DF.csv"))
On Mon, Mar 29, 2010 at 8:33 PM, D
To be specific, my data frame is 4000 by 2200.
On Mon, Mar 29, 2010 at 8:33 PM, Dimitri Liakhovitski wrote:
> Just tried it. It's definitely faster - but I get the same error:
> " Reached total allocation of 1535Mb:"
>
> On Mon, Mar 29, 2010 at 8:27 PM, Gabor Grothendieck
> wrote:
>> See if this
Just tried it. It's definitely faster - but I get the same error:
" Reached total allocation of 1535Mb:"
On Mon, Mar 29, 2010 at 8:27 PM, Gabor Grothendieck
wrote:
> See if this works for you:
>
> DF[is.na(DF)] <- 0
>
> On Mon, Mar 29, 2010 at 8:21 PM, Dimitri Liakhovitski
> wrote:
>> Dear R'er
See if this works for you:
DF[is.na(DF)] <- 0
On Mon, Mar 29, 2010 at 8:21 PM, Dimitri Liakhovitski wrote:
> Dear R'ers,
>
> I have a very large data frame (over 4000 rows and 2,500 columns). My
> task is very simple - I have to replace all NAs with a zero. My code
> works fine on smaller data f
Dear R'ers,
I have a very large data frame (over 4000 rows and 2,500 columns). My
task is very simple - I have to replace all NAs with a zero. My code
works fine on smaller data frames - but I have to deal with a huge one
and there are many NAs in each column.
R runs out of memory on me ("Reached
13 matches
Mail list logo
