> PS. Simon, I think your explicit comment on mcparallel() & friends is
very helpful for many people and developers. It clearly tells
developers to never use mclapply() as the only path through their
code. I'm quite sure not everyone has been or is aware of this. Now
it's clear. Thank you.
I secon
On Tue, Apr 28, 2020 at 9:00 PM Shian Su wrote:
>
> Thanks Simon,
>
> I will take note of the sensible default for core usage. I’m trying to
> achieve small scale parallelism, where tasks take 1-5 seconds and make fuller
> use of consumer hardware. Its not a HPC-worthy computation but even lapto
Thanks Simon,
I will take note of the sensible default for core usage. I’m trying to achieve
small scale parallelism, where tasks take 1-5 seconds and make fuller use of
consumer hardware. Its not a HPC-worthy computation but even laptops these days
come with 4 cores and I don’t see a reason to
Do NOT use mcparallel() in packages except as a non-default option that user
can set for the reasons Henrik explained. Multicore is intended for HPC
applications that need to use many cores for computing-heavy jobs, but it does
not play well with RStudio and more importantly you don't know the r
Thanks Henrik,
That clears things up significantly. I did see the warning but failed to
include it my initial email. It sounds like an RStudio issue, and it seems like
that it’s quite intrinsic to how forks interact with RStudio. Given this code
is eventually going to be a part of a package, sh
Hi, a few comments below.
First, from my experience and troubleshooting similar reports from
others, a returned NULL from parallel::mclapply() is often because the
corresponding child process crashed/died. However, when this happens
you should see a warning, e.g.
> y <- parallel::mclapply(1:2, FU
Yes I am running on Rstudio 1.2.5033. I was also running this code without
error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed
work fine even with large data.frames.
Any idea as to what interaction between Rstudio and mclapply causes this?
Thanks,
Shian
On 28 Apr 2020
Sorry, the code works perfectly fine for me in R even for 1e6 observations (but
I was testing with R 4.0.0). Are you using some kind of GUI?
Cheers,
Simon
> On 28/04/2020, at 8:11 PM, Shian Su wrote:
>
> Dear R-devel,
>
> I am experiencing issues with running GAM models using mclapply, it fa
Dear R-devel,
I am experiencing issues with running GAM models using mclapply, it fails to
return any values if the data input becomes large. For example here the code
runs fine with a df of 100 rows, but fails at 1000.
library(mgcv)
library(parallel)
> df <- data.frame(
+ x = 1:100,
+
