Hi,
I used to run a hello mpi for testing purposes. Now, I see that it doesn't
work. While the log file shows memory allocation problem, squeue shows that
job is in R state endlessly.
[mahmood@hpc ~]$ cat slurm_script1.sh
#!/bin/bash
#SBATCH --job-name=hello_mpi
#SBATCH --output=hellompi.log
#SBATCH --ntasks=4
#SBATCH --time=10:00
#SBATCH --partition=SEA
#SBATCH --account=fish
#SBATCH --mem=100M
mpirun ./mpihello
[mahmood@hpc ~]$ sbatch slurm_script1.sh
Submitted batch job 10
[mahmood@hpc ~]$ cat hellompi.log
[hpc.safaar.com:18059] create_and_attach: unable to create shared memory
BTL coordinating structure :: size 134217728
--------------------------------------------------------------------------
A system call failed during shared memory initialization that should
not have. It is likely that your MPI job will now either abort or
experience performance degradation.
Local host: hpc.safaar.com
System call: mmap(2)
Error: Cannot allocate memory (errno 12)
--------------------------------------------------------------------------
[mahmood@hpc ~]$ squeue
JOBID PARTITION NAME USER ST TIME NODES
NODELIST(REASON)
10 SEA hello_mp mahmood R 0:36 1 hpc
[mahmood@hpc ~]$ squeue
JOBID PARTITION NAME USER ST TIME NODES
NODELIST(REASON)
10 SEA hello_mp mahmood R 0:43 1 hpc
[mahmood@hpc ~]$ sacctmgr list association
format=partition,account,user,grptres
Partition Account User GrpTRES
---------- ---------- ---------- -------------
root
root root
fish
sea fish mahmood cpu=10,mem=8G
local mahmood
However, the binary file works fine outside of slurm.
Regards,
Mahmood
