we do exactly what you describe: compile OpenMPI with the gcc suite
and then use it with gcc, intel and open64 compilers.
nice.
This works out-of-the-box, almost.
The problem is the f90 module mpi.mod. This is (usually) a binary
file and specific to the f90 compiler that was used to compile
OpenMPI. But there is a way to solve this problem:
ah, .mod files. we have for the most part ignored them entirely.
(what are we losing by doing that?)
1. compile openmpi using the gcc compilers, i.e., gfortran as the Fortran
compiler and install it in /usr/local/openmpi
this is perhaps a tangent, but we install everything we support
under /opt/sharcnet/$packagebasename/$ver. for openmpi, we've had
to bodge the compiler flavor onto that (/opt/sharcnet/openmpi/1.4.2/intel).
2. move the Fortran module to the directory
/usr/local/openmpi/include/gfortran. In that directory
create softlinks to the files in /usr/local/openmpi/include.
3. compile openmpi using ifort and install the Fortran module (and only
the Fortran module!) in /usr/local/openmpi/include/intel. In that
directory create softlinks to the files in /usr/local/openmpi/include.
I guess I'm surprised this works - aren't you effectively assuming that
the intel and gfortran interfaces are compatible here? that is, the app
compiles with the compiler-specific module, which basically promises
a particular type-safe interface (signature) for MPI functions, but
then the linker just glues them together without any way to verify
the signature compatibility...
am I misunderstanding?
4. in /usr/local/openmpi/bin create softlinks mpif90.ifort
and mpif90.gfortran pointing to opal_wrapper. Remove the
mpif90 softlink.
5. Move /usr/local/openmpi/share/openmpi/mpif90-wrapper-data.txt
to /usr/local/openmpi/share/openmpi/mpif90.ifort-wrapper-data.txt.
Change the line includedir=${includedir} to:
includedir=${includedir}/intel
Copy the file to
/usr/local/openmpi/share/openmpi/mpif90.gfortran-wrapper-data.txt
and change the line includedir=${includedir} to
includedir=${includedir}/gfortran
6. Create a wrapper script /usr/local/openmpi/bin/mpif90:
#!/bin/bash
OMPI_WRAPPER_FC=`basename $OMPI_FC 2> /dev/null`
if [ "$OMPI_WRAPPER_FC" = 'gfortran' ]; then
exec $0.gfortran "$@"
else
exec $0.ifort "$@"
fi
this is a tangent, but perhaps interesting. we don't use the wrappers
from the MPI package, but rather our own single wrapper which has some
built-in intelligence (augmented by info from the compiler's (environment)
module.)
The reason we use gfortran in step 1 is that otherwise you get those
irritating error messages from the Intel libraries, cf.
http://www.open-mpi.org/faq/?category=building#intel-compiler-wrapper-compiler-warnings
hmm. we work around those by manipulating the link arguments in our wrapper.
thanks!
-mark
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
