Dear all,
I’m working as an administrator of a High-Performance Computing (HPC) Cluster
which runs on Linux. A lot of people are using R on this Linux cluster and, of
course, the *parallel* package to speed up their computations.
It has been our collective experience, that using |makeForkCluster| yields an
overall better experience /on Linux/ than the |makePSOCKcluster|, for whatever
definition of better. Let me just summarize that it works smoother. I believe,
other people working with *parallel* on Linux can share this experience
Also, we did really welcome the environment variable |MC_CORES|, to be able to
specify (in job submit scripts) the amount of CPU cores a user has been
granted, most importantly for /dynamic resource requests/ (see below for an
example).
What we would also appreciate - and now we finally get to the feature request -
is another environment variable to choose the used cluster, as in:
|export MC_CLUSTER_TYPE=FORK |
Do you think something like this could be implemented in future releases?
Parallel R job submit script
This works with the Univa Grid Engine and should work with other * Grid Engine
products:
|#!/bin/bash # request a "parallel environment" with 2 to 20 cores #$ -pe smp
2-20 # set number of cores for the R cluster to the granted value (between 2
and 20) export MC_CORES=$NSLOTS # we want this: export MC_CLUSTER_TYPE=FORK
Rscript /path/to/script.R |
Best Regards
--
Christian Krause
Scientific Computing Administration and Support
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Phone: +49 341 97 33144
Email: [email protected]
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig
Deutscher Platz 5e
04103 Leipzig
Germany
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
iDiv is a research centre of the DFG – Deutsche Forschungsgemeinschaft
iDiv ist eine zentrale Einrichtung der Universität Leipzig im Sinne des § 92
Abs. 1 SächsHSFG und wird zusammen mit der Martin-Luther-Universität
Halle-Wittenberg und der Friedrich-Schiller-Universität Jena betrieben sowie in
Kooperation mit dem Helmholtz-Zentrum für Umweltforschung GmbH – UFZ.
Beteiligte Kooperationspartner sind die folgenden außeruniversitären
Forschungseinrichtungen: das Helmholtz-Zentrum für Umweltforschung GmbH - UFZ,
das Max-Planck-Institut für Biogeochemie (MPI BGC), das Max-Planck-Institut für
chemische Ökologie (MPI CE), das Max-Planck-Institut für evolutionäre
Anthropologie (MPI EVA), das Leibniz-Institut Deutsche Sammlung von
Mikroorganismen und Zellkulturen (DSMZ), das Leibniz-Institut für
Pflanzenbiochemie (IPB), das Leibniz-Institut für Pflanzengenetik und
Kulturpflanzenforschung (IPK) und das Leibniz-Institut Senckenberg Museum für
Naturkunde Görlitz (SMNG). USt-IdNr. DE 141510383
[[alternative HTML version deleted]]
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
