Re: [Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

2018-10-05 Thread Seth Russell
Jeroen/Tomas, Thanks for the additional follow-ups. As shown in my original post, I am on macOS 10.12.6. I was using the command line version of R to get the error message I sent; I also get similar problems when running via GUI tools such as RStudio. I have co-workers who are only using macOS and

Re: [Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

2018-10-04 Thread Jeroen Ooms
On Thu, Oct 4, 2018 at 6:12 PM Tomas Kalibera wrote: > > > Thanks for the report, but unfortunately I cannot reproduce on my system > (either macOS nor Linux, from the command line) to debug. Did you run > this in the command line version of R? It depends on which version of MacOS that you are us

Re: [Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

2018-10-04 Thread Tomas Kalibera
Thanks for the report, but unfortunately I cannot reproduce on my system (either macOS nor Linux, from the command line) to debug. Did you run this in the command line version of R? I would not be surprised to see such a crash if executed from a multi-threaded application, say from some GUI

Re: [Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

2018-09-20 Thread Seth Russell
Thanks for the warning about fork without exec(). A co-worker of mine, also on Mac, ran the sample code and got an error about that exact problem. Thanks also for the pointer to try curl::multi_add() or download.file() with a vector of files. My actual use case includes downloading the files and

Re: [Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

2018-09-20 Thread Gábor Csárdi
This code actually happens to work for me on macOS, but I think in general you cannot rely on performing HTTP requests in fork clusters, i.e. with mclapply(). Fork clusters create worker processes by forking the R process and then _not_ executing another R binary. (Which is often convenient, becau

Re: [Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

2018-09-20 Thread Martin Maechler
> Seth Russell > on Wed, 19 Sep 2018 15:19:48 -0600 writes: > I have an lapply function call that I want to parallelize. Below is a very > simplified version of the code: > url_base <- "https://cloud.r-project.org/src/contrib/"; > files <- c("A3_1.0.0.tar.gz", "ABC.RA

[Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

2018-09-20 Thread Seth Russell
I have an lapply function call that I want to parallelize. Below is a very simplified version of the code: url_base <- "https://cloud.r-project.org/src/contrib/"; files <- c("A3_1.0.0.tar.gz", "ABC.RAP_0.9.0.tar.gz") res <- parallel::mclapply(files, function(s) download.file(paste0(url_base, s), s