One more note. In case it is helpful, I am including the code for my loop:
# data is read in
numSNPs <- ncol(myData);
pvalues <- rep(-1,numSNPs);
names(pvalues) <- colnames(myData);
for (SNPnum in 1:numSNPs)
{
is.na(pvalues[SNPnum]) <- TRUE;
try({
fit.yags <- yags(log(myPhenos$PHENOTYPE) ~
myPhenos$AGE+myPhenos$SEX*myData[,SNPnum], id=myPhenos$id,
family=gaussian,corstr="exchangeable",alphainit=0.05)
z.gee <- fit.y...@coefficients[5]/sqrt(fit.y...@robust.parmvar[5,5]);
pval <- 2 * pnorm(abs(z.gee), lower.tail = FALSE);
pvalues[SNPnum] <- pval;
})
}
pvalues <- format(pvalues,digits=3);
On Mon, Dec 29, 2008 at 11:59 AM, Juliet Hannah <juliet.han...@gmail.com> wrote:
> I monitored the usage of memory on a script that I ran. It ran 30K
> regressions and it stores p-values for one of the
> coefficients. It read in a file that has 3000 rows and about 30K
> columns. The size of the file is about 170 MB.
>
> My understanding is that memory usage started out at 2.2G and went up to 23G:
>
>
> cpu=00:03:08, mem=172.75822 GBs, io=0.00000, vmem=2.224G, maxvmem=2.224G
> cpu=00:42:35, mem=29517.64894 GBs, io=0.00000, vmem=23.612G, maxvmem=23.612G
>
> I know very little about how memory works, but I thought the hardest
> part would be reading the file in. Could
> someone explain why there is such a substantial increase over the
> course of the script.
>
> Thanks,
>
> Juliet
>
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
