Re: [Rd] SUGGESTION: Settings to disable forked processing in R, e.g. parallel::mclapply()

2019-04-15 Thread Iñaki Ucar
On Mon, 15 Apr 2019 at 08:44, Tomas Kalibera  wrote:
>
> On 4/13/19 12:05 PM, Iñaki Ucar wrote:
> > On Sat, 13 Apr 2019 at 03:51, Kevin Ushey  wrote:
> >> I think it's worth saying that mclapply() works as documented
> > Mostly, yes. But it says nothing about fork's copy-on-write and memory
> > overcommitment, and that this means that it may work nicely or fail
> > spectacularly depending on whether, e.g., you operate on a long
> > vector.
>
> R cannot possibly replicate documentation of the underlying operating
> systems. It clearly says that fork() is used and readers who may not
> know what fork() is need to learn it from external sources.
> Copy-on-write is an elementary property of fork().

Just to be precise, copy-on-write is an optimization widely deployed
in most modern *nixes, particularly for the architectures in which R
usually runs. But it is not an elementary property; it is not even
possible without an MMU.

-- 
Iñaki Úcar

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


Re: [Rd] SUGGESTION: Settings to disable forked processing in R, e.g. parallel::mclapply()

2019-04-15 Thread Tomas Kalibera

On 4/15/19 11:02 AM, Iñaki Ucar wrote:

On Mon, 15 Apr 2019 at 08:44, Tomas Kalibera  wrote:

On 4/13/19 12:05 PM, Iñaki Ucar wrote:

On Sat, 13 Apr 2019 at 03:51, Kevin Ushey  wrote:

I think it's worth saying that mclapply() works as documented

Mostly, yes. But it says nothing about fork's copy-on-write and memory
overcommitment, and that this means that it may work nicely or fail
spectacularly depending on whether, e.g., you operate on a long
vector.

R cannot possibly replicate documentation of the underlying operating
systems. It clearly says that fork() is used and readers who may not
know what fork() is need to learn it from external sources.
Copy-on-write is an elementary property of fork().

Just to be precise, copy-on-write is an optimization widely deployed
in most modern *nixes, particularly for the architectures in which R
usually runs. But it is not an elementary property; it is not even
possible without an MMU.


Yes, old Unix systems without virtual memory had fork eagerly copying. 
Not relevant today, and certainly not for systems that run R, but indeed 
people interested in OS internals can look elsewhere for more precise 
information.


Tomas

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel