Re: [Rd] Reducing RAM usage using UKSM

2014-07-16 Thread Varadharajan Mukundan
Hi Gregory, > Linux uses a copy-on-write for the memory image of forked processes. Thus, > you may also get significant memory savings by launching a single R process, > loading your large data object, and then using fork::fork() to split off the > other worker process. True. The problem is e

Re: [Rd] Reducing RAM usage using UKSM

2014-07-16 Thread Varadharajan Mukundan
Hi, > But Linux copied it from Unix and I see no mention of Linux in the posting > being replied to. Yes. As Gregory mentioned, UKSM is implemented on Linux as a patch. Sorry for the in-direct reference. - Varadharajan __ R-devel@r-project.org mailin

Re: [Rd] Reducing RAM usage using UKSM

2014-07-16 Thread Varadharajan Mukundan
tions that we should be considering? On Tue, Jul 15, 2014 at 9:25 PM, Varadharajan Mukundan wrote: > Greetings, > > We've a fairly large dataset (around 60GB) to be loaded and crunched in real > time. The kind of data operations that will be performed on this data are > sim

[Rd] Reducing RAM usage using UKSM

2014-07-15 Thread Varadharajan Mukundan
Greetings, We've a fairly large dataset (around 60GB) to be loaded and crunched in real time. The kind of data operations that will be performed on this data are simple read only aggregates after filtering the data.table instance based on the parameters that will passed in real time. We need to ha