Here I just want to know why cannot I use .GlobalEnv$y to access y in parLapply?
parLapply does not attempt to make the GlobalEnv's of all the subprocesses
identical.
Many times that would not be appropriate, either because you want the various
workers
to do different things or because it would take too much time. My trick worked
because
it appears to send the function FUN to the workers along with FUN's environment
and
its ancestral environments, up to but not including .GlobalEnv, the same as the
save() function
does.
By the way, my function should have a parent= argument in its call to list2env()
so its own environment (which includes 'FUN' and '...') doesn't get added to
FUN's
environment list.
withGlobals <- function(FUN, ..., envir=parent.frame()){
environment(FUN) <- list2env(list(...), parent=envir)
FUN
}
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
From: Yu Wan [mailto:walter...@126.com]
Sent: Friday, December 27, 2013 3:46 AM
To: William Dunlap; alex...@4dscape.com
Cc: r-devel@r-project.org
Subject: Re: [Rd] Parallel computing: how to transmit multiple parameters to a
function in parLapply?
Thanks to William and Alexios, both of your solution worked well and solved my
problem.
Here I just want to know why cannot I use .GlobalEnv$y to access y in parLapply?
Best regards,
Yu
On 24/12/2013 11:20 PM, alexios ghalanos wrote:
This works:
clusterExport(cl, c("f","y"), envir=environment())
r <- parLapply(cl, x, function(x) f(x,y))
You need to export your function ("f") and additional variables ("y"), and then
define that function inside parLapply ("f(x,y)"). If you were to also make use
of
additional libraries (or source some scripts) then you should also consult
"clusterEvalQ".
The makeCluster command (at least in windows via socket) just initializes new R
processes which do not know about your functions or variables unless you
export those to them.
Perhaps a question best suited for R-help.
Alexios
On 25/12/2013 2:57 AM, William Dunlap wrote:
You can put the function of interest and any global
variables it needs into a private environment, which gets sent
along with the function to the child processes. E.g.
library(parallel)
cl3 <- makeCluster(3)
y <- c(1,100,10000)
addY <- function(x) x + y
withGlobals <- function(FUN, ...){
environment(FUN) <- list2env(list(...))
FUN
}
parLapply(cl3, 1:4, withGlobals(addY, y=y))
# [[1]]
# [1] 2 101 10001
#
# [[2]]
# [1] 3 102 10002
# ...
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
-----Original Message-----
From: r-devel-boun...@r-project.org<mailto:r-devel-boun...@r-project.org>
[mailto:r-devel-boun...@r-project.org] On Behalf
Of Yu Wan
Sent: Monday, December 23, 2013 10:16 PM
To: r-devel@r-project.org<mailto:r-devel@r-project.org>
Subject: [Rd] Parallel computing: how to transmit multiple parameters to a
function in
parLapply?
Hi R-developers
In the package Parallel, the function parLapply(cl, x, f) seems to allow
transmission of only one parameter (x) to the function f. Hence in order to
compute f(x, y) parallelly, I had to define f(x, y) as f(x) and tried to
access y within the function, whereas y was defined outside of f(x).
Script:
library(parallel)
f <- function(x) {
z <- 2 * x + .GlobalEnv$y # Try to access y in the global scope.
return(z)
}
np <- detectCores(logical = FALSE) # Two cores of my laptop
x <- seq(1, 10, by = 1)
y <- 0.5 # Y may be an array in reality.
cl <- makeCluster(np) # initiate the cluster
r <- parLapply(cl, x, f) # apply f to x for parallel computing
stopCluster(cl)
The r was a list with 10 empty elements which means f failed to access y.
Then I tested f without parallel computing:
z <- f(x)
print(z)
[1] 2.5 4.5 6.5 8.5 10.5 12.5 14.5 16.5 18.5 20.5
The results indicates that we can access y using .GlobalEnv$y in a function
without parLapply.
The question is, is there any method for me to transmit y to f, or access y
within f during parallel computing?
The version of my R is 3.0.1 and I am running it on a Win8-64x system.
Thanks,
Yu
--
View this message in context:
http://r.789695.n4.nabble.com/Parallel-computing-how-
to-transmit-multiple-parameters-to-a-function-in-parLapply-tp4682667.html
Sent from the R devel mailing list archive at Nabble.com.
______________________________________________
R-devel@r-project.org<mailto:R-devel@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[[alternative HTML version deleted]]
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
