On 01 Jul 2014, at 03:27, Christopher Samuel <sam...@unimelb.edu.au> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 26/06/14 05:08, Kilian Cavalotti wrote:
>
>> You can just use a very minimal OS on your compute nodes, then
>> compile and install all the user facing bits in a shared location.
>> You hand an environment modules system to the users and off they
>> go.
>
> That's pretty much exactly what I've done for the last 8 years or so,
> and it works really well.
>
> Our current incarnation is to netboot a minimal RHEL image on diskless
> nodes (managed via xCAT) and then all the applications are in
> /usr/local including various Perl, Python and R versions with their
> associated module hell. At the moment we've got about 638 different
> modules across 3 different architectures.
>
> The main reason for RHEL on compute nodes is to keep hardware support
> people happy, most vendors take the Blue Brothers approach to Linux
> ("we support both kinds of Linux, RHEL *and* SLES”).
This is especially true in an environment where you have, or plan to
have parallel filesystems, InfiniBand, accelerators and libraries tightly
coupled
to them (say, HDF5+Lustre w/RDMA or Tesla+MPI w/GPUDirect). In these cases
you end up with a very small subset of “common denominator” distros+versions
that you can deal with. RHEL/CentOS 6.[N-1] (where N is the latest version)
seems
to be a relatively safe bet.
It is possible to build things on your own but you easily end up in very
uncharted
territory. In this kind of an environment implementing Docker and/or VMs is
also
quite a bit more challenging.
On a sidenote, EasyBuild was mentioned here earlier and it seems that they
have IMO right idea of simplifying installs of the type of environment that
Chris
was describing and many (including us) seem to hold as best practice. Just
heard
about it at ISC and haven’t had the chance to test it in practice yet, though.
O-P
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
