Re: [Rd] Memory limitations for parallel::mclapply

2015-07-24 Thread Ista Zahn
Hi Josh, I think we need some more details, including code, and information about your operating system. My machine has only 12 Gb of ram, but I can run this quite comfortably (no swap, other processes using memory etc.): library(parallel) library(data.table) d <- data.table(a = rnorm(5000),

[Rd] Memory limitations for parallel::mclapply

2015-07-24 Thread Joshua Bradley
Hello, I have been having issues using parallel::mclapply in a memory-efficient way and would like some guidance. I am using a 40 core machine with 96 GB of RAM. I've tried to run mclapply with 20, 30, and 40 mc.cores and it has practically brought the machine to a standstill each time to the poin