Re: [Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Henrik Bengtsson
On Tue, Apr 28, 2020 at 9:00 PM Shian Su wrote: > > Thanks Simon, > > I will take note of the sensible default for core usage. I’m trying to > achieve small scale parallelism, where tasks take 1-5 seconds and make fuller > use of consumer hardware. Its not a HPC-worthy computation but even lapto

Re: [Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Shian Su
Thanks Simon, I will take note of the sensible default for core usage. I’m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don’t see a reason to

Re: [Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Simon Urbanek
Do NOT use mcparallel() in packages except as a non-default option that user can set for the reasons Henrik explained. Multicore is intended for HPC applications that need to use many cores for computing-heavy jobs, but it does not play well with RStudio and more importantly you don't know the r

Re: [Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Shian Su
Thanks Henrik, That clears things up significantly. I did see the warning but failed to include it my initial email. It sounds like an RStudio issue, and it seems like that it’s quite intrinsic to how forks interact with RStudio. Given this code is eventually going to be a part of a package, sh

[Rd] R 4.0.0 build error with sysdata.rda on ppc64el architecture

2020-04-28 Thread Dirk Eddelbuettel
The R 4.0.0 package migration on Debian is being held back by a failed build on ppc64el [1]. We can see from the history of builds logs [2] that it used to build, briefly failed, worked again and then failed leading to R 4.0.0's release. (And my bad for missing how the alpha1/alpha2/beta/rc build

Re: [Rd] Rtools and R 4.0.0?

2020-04-28 Thread Avraham Adler
Absolutely; this is a complicated and frustrating procedure, and we owe Jeoren and all our gratitude! Avi On Tue, Apr 28, 2020 at 3:37 PM Gabriel Becker wrote: > > Huge thanks to you (Jeroen) and R-core for doing this. > > I wasn't involved with this directly but I know it was a pretty seriousl

Re: [Rd] Rtools and R 4.0.0?

2020-04-28 Thread Gabriel Becker
Huge thanks to you (Jeroen) and R-core for doing this. I wasn't involved with this directly but I know it was a pretty seriously heavy list so well done all around! ~G On Tue, Apr 28, 2020, 11:04 AM Hervé Pagès wrote: > Thanks Jeroen! > > > On Tue, Apr 7, 2020 at 6:07 PM Kevin Ushey wrote:

Re: [Rd] Rtools and R 4.0.0?

2020-04-28 Thread Hervé Pagès
Thanks Jeroen! On Tue, Apr 7, 2020 at 6:07 PM Kevin Ushey wrote: Regardless, I would like to thank R core, CRAN, and Jeroen for all of the time that has gone into creating and validating this new toolchain. This is arduous work at an especially arduous time, so I'd like to voice my appreciati

Re: [Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Henrik Bengtsson
Hi, a few comments below. First, from my experience and troubleshooting similar reports from others, a returned NULL from parallel::mclapply() is often because the corresponding child process crashed/died. However, when this happens you should see a warning, e.g. > y <- parallel::mclapply(1:2, FU

Re: [Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Shian Su
Yes I am running on Rstudio 1.2.5033. I was also running this code without error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed work fine even with large data.frames. Any idea as to what interaction between Rstudio and mclapply causes this? Thanks, Shian On 28 Apr 2020

Re: [Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Simon Urbanek
Sorry, the code works perfectly fine for me in R even for 1e6 observations (but I was testing with R 4.0.0). Are you using some kind of GUI? Cheers, Simon > On 28/04/2020, at 8:11 PM, Shian Su wrote: > > Dear R-devel, > > I am experiencing issues with running GAM models using mclapply, it fa

[Rd] mclapply returns NULLs on MacOS when running GAM

2020-04-28 Thread Shian Su
Dear R-devel, I am experiencing issues with running GAM models using mclapply, it fails to return any values if the data input becomes large. For example here the code runs fine with a df of 100 rows, but fails at 1000. library(mgcv) library(parallel) > df <- data.frame( + x = 1:100, +