Re: [R] Thread parallelism and memory management on shared-memory supercomputers

2015-12-30 Thread Peter Langfelder
I'm not really an expert, but here are my 2 cents: To the best of my limited knowlede, there is no direct way of ensuring that the total memory being requested by N workers remains below a certain threshold. You can control the number of child processes forked by foreach/doPar in the registerDoPar

[R] Thread parallelism and memory management on shared-memory supercomputers

2015-12-30 Thread Andrew Crane-Droesch
I've got allocations on a couple of shared memory supercomputers, which I use to run computationally-intensive scripts on multiple cores of the same node. I've got 24 cores on the one, and 48 on the other. In both cases, there is a hard memory limit, which is shared among the cores in the nod