Re: [Rd] Environments and parallel processing

2018-09-13 Thread Clark Fitzgerald
+1 to what Gabor and Ralf said. In this case the memory address can be misleading. My understanding is that the environments in all the processes, 1 parent and 2 child, have the *same* memory address, but once you write to them the operating system makes copies and maps the address to a *different

Re: [Rd] Environments and parallel processing

2018-09-12 Thread Ralf Stubner
On 12.09.2018 20:20, Gábor Csárdi wrote: > This is all normal, a fork cluster works with processes, that do not > share memory. And if you are after shared-memory parallelism, you can try the 'Rdsm' package: https://cran.r-project.org/package=Rdsm Greetings Ralf -- Ralf Stubner Senior Software

Re: [Rd] Environments and parallel processing

2018-09-12 Thread Gábor Csárdi
This is all normal, a fork cluster works with processes, that do not share memory. When you create a fork cluster, you create a new process, that has the same memory layout as the parent. But from this moment its memory is independent of the parent process. When parLapply is done, the results are s

[Rd] Environments and parallel processing

2018-09-12 Thread Niek Bouman
While using parallelization R seems to clone all environments (that are normally passed by reference) that are returned from a child process. In particular, consider the following example: library(parallel) env1 <- new.env() envs2 <- lapply(1:4, function(x) env1) cl<-makeCluster(2, type="FORK")