Hi Simon,
I'm having trouble with nested parallel workers, specifically, forking
inside socket connections.
When mclapply is called inside a SOCK, PSOCK or FORK worker I get an
error in unserialize().
cl <- makeCluster(1, "SOCK")
fun = function(i) {
library(parallel)
mclapply(1:2, sqrt)
}
Failure occurs after multiple calls to clusterApply:
> clusterApply(cl, 1, fun)
[[1]]
[[1]][[1]]
[1] 1
[[1]][[2]]
[1] 1.414214
> clusterApply(cl, 1, fun)
[[1]]
[[1]][[1]]
[1] 1
[[1]][[2]]
[1] 1.414214
> clusterApply(cl, 1, fun)
Error in unserialize(node$con) : error reading from connection
This example is from Martin and may be a different problem.
~/tmp >cat test1.R
## like mclapply
## should run 'forever' but terminates semi-randomly
library(parallel)
children <- parallel:::children
while (TRUE) {
n <- 8 ## n == dectectCores()
jobs <- lapply(seq_len(n), function(i) mcparallel(Sys.sleep(20)))
mccollect(children(jobs), FALSE)
parallel:::mckill(children(jobs), tools::SIGTERM)
leni <- length(mccollect(children(jobs)))
message("leni: ", leni)
}
~/tmp >R-dev --vanilla --slave -f test1.R
leni: 6
leni: 7
leni: 7
leni: 7
leni: 7
leni: 7
leni: 7
leni: 7
leni: 8
leni: 7
leni: 7
leni: 7
~/tmp >
Thanks.
Valerie
> sessionInfo()
R Under development (unstable) (2015-03-18 r68009)
Platform: x86_64-unknown-linux-gnu (64-bit)
Running under: Fedora 21 (Twenty One)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
loaded via a namespace (and not attached):
[1] snow_0.3-13
--
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, Seattle, WA 98109
Email: voben...@fredhutch.org
Phone: (206) 667-3158
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
