Shouldn't -n be 4 in the bsub command?
One master+3 slaves. This was required for snowfall, but I think doSNOW
is similar.
Hope it helps
mario
On 16-Dec-10 23:09, Marius Hofert wrote:
Dear expeRts,
I try to get a minimal example for parallel computing via "foreach" + "doSNOW"
to run on a computer cluster (Brutus from ETH Zurich). The minimal example is given below. It runs
perfectly fine on my MacBook but when I submit it as a batch job via ...
bsub -n 3 -R "select[model==Opteron8380]" mpirun R --no-save -q -f
doSNOW_minimal.R
... it does not work. The output is also given below. The error is "Error in
makeMPIcluster(spec, ...) : a cluster already exists 1". The only similar thing I
found on the web is http://www.mail-archive.com/r-help@stat.math.ethz.ch/msg35501.html
but unfortunately, it was not answered.
I also contacted the maintainers of the cluster, however, they couldn't really help me
since they were no "expeRts".
Cheers,
Marius
Ps: As you an expeRt probably sees right away, I am a newby when it comes to parallel
computing, so please check all the details (does the mpirun command makes sense? and the
bsubs command [the batch system is "LSF"]?) and let me know if you need further.
## ==== minimal example ====
library(doSNOW) # loads foreach
library(Rmpi) # for default in makeCluster()
cl<- makeCluster(3) # create cluster object with the given number of slaves
registerDoSNOW(cl) # register the cluster object with foreach
x<- foreach(i = 1:3) %dopar% { # simple test
sqrt(i)
}
x
stopCluster(cl) # properly shut down the cluster
## ==== minimal example ====
## ==== output ====
Sender: LSF System<lsfad...@a6218>
Subject: Job 921476:<mpirun R --no-save -q -f doSNOW_minimal.R> Done
Job<mpirun R --no-save -q -f doSNOW_minimal.R> was submitted from host<brutus2> by
user<hofertj> in cluster<brutus>.
Job was executed on host(s)<3*a6218>, in queue<pub.1h>, as user<hofertj> in
cluster<brutus>.
</cluster/home/math/hofertj> was used as the home directory.
</cluster/home/math/hofertj> was used as the working directory.
Started at Thu Dec 16 22:16:31 2010
Results reported at Thu Dec 16 22:16:43 2010
Your job looked like:
------------------------------------------------------------
# LSBATCH: User input
mpirun R --no-save -q -f doSNOW_minimal.R
------------------------------------------------------------
Successfully completed.
Resource usage summary:
CPU time : 6.96 sec.
Max Memory : 3 MB
Max Swap : 29 MB
Max Processes : 1
Max Threads : 1
The output (if any) follows:
master (rank 0, comm 1) of size 3 is running on: a6218
slave1 (rank 1, comm 1) of size 3 is running on: a6218
slave2 (rank 2, comm 1) of size 3 is running on: a6218
library(doSNOW) # loads foreach
Loading required package: foreach
Loading required package: iterators
Loading required package: codetools
Loading required package: snow
library(Rmpi) # for default in makeCluster()
cl<- makeCluster(3) # create cluster object with the given number of slaves
Error in makeMPIcluster(spec, ...) : a cluster already exists 1
Calls: makeCluster -> makeMPIcluster
registerDoSNOW(cl) # register the cluster object with foreach
Error in assign("data", data, pos = .foreachGlobals, inherits = FALSE) :
object 'cl' not found
Calls: registerDoSNOW -> setDoPar -> assign
x<- foreach(i = 1:3) %dopar% { # simple test
+ sqrt(i)
+ }
Error in checkCluster(cl) : not a valid cluster
Calls: %dopar% -> <Anonymous> -> clusterCall -> checkCluster
x
Error: object 'x' not found
stopCluster(cl) # properly shut down the cluster
Error in stopCluster(cl) : object 'cl' not found
[1] "Please use mpi.close.Rslaves() to close slaves"
[1] "Please use mpi.quit() to quit R"
## ==== output ====
_______________________________________________
R-sig-hpc mailing list
r-sig-...@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
--
Ing. Mario Valle
Data Analysis and Visualization Group | http://www.cscs.ch/~mvalle
Swiss National Supercomputing Centre (CSCS) | Tel: +41 (91) 610.82.60
v. Cantonale Galleria 2, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
