Re: [slurm-users] Running pyMPI on several nodes

2019-07-11 Thread John Hearns
Please try something very simple such as a hello world program or srun -N2 -n8 hostname What is the error message which you have ? On Fri, 12 Jul 2019 at 07:07, Pär Lundö wrote: > > Hi there Slurm-experts! > I am trouble using or running a python-mpi program involving more than > one node. The

Re: [slurm-users] Running pyMPI on several nodes

2019-07-11 Thread John Hearns
MY apology. You do say that the Python program simply printe the rank - so is a hello world program. On Fri, 12 Jul 2019 at 07:45, John Hearns wrote: > Please try something very simple such as a hello world program or > srun -N2 -n8 hostname > > What is the error message which you have ? > > On

[slurm-users] Running pyMPI on several nodes

2019-07-11 Thread Pär Lundö
Hi there Slurm-experts! I am trouble using or running a python-mpi program involving more than one node. The pythom-mpi program is very simple, it only lists the number of ranks that is available in its environment. I have a munge-daemon running prior to starting the slurm-service and the prog

Re: [slurm-users] Hide Filesystem From Slurm

2019-07-11 Thread Douglas Duckworth
Thank you everyone! We will Spank it first Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: d...@med.cornell.edu O: 212-746-6305 F: 212-7

[slurm-users] Slurm showing 100% utilization since last maintenance window

2019-07-11 Thread Prentice Bisbal
I have a strange issue: sreport is showing 100% utilization for our cluster every day since June 18. What is interesting about this is June 18th was our last maintenance outage, when all the nodes were rebooted, including our slurm server which runs both slurmdbd and slurmctld. Has anyone else

Re: [slurm-users] Hide Filesystem From Slurm

2019-07-11 Thread Lloyd Brown
We've taken the per-user-namespace approach, as my boss documented here: http://tech.ryancox.net/2013/07/per-user-tmp-and-devshm-directories.html This setup makes sure that all user processes are in their own namespace, whether they're launched by slurm or not.  We do put them inside a subdir of /

Re: [slurm-users] Hide Filesystem From Slurm

2019-07-11 Thread Ryan Novosielski
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There are some plugins of one sort or another (at least one is SPANK) that can create a private temporary directory. We've been considering implementing something like this but have not gotten around to it. "systemd" also offers some of this functional

Re: [slurm-users] Hide Filesystem From Slurm

2019-07-11 Thread Christopher Samuel
On 7/11/19 8:19 AM, Douglas Duckworth wrote: I am wondering if it's possible to hide a file system, that's world writable on compute node, logically within Slurm.  That way any job a user runs cannot possible access this file system. Essentially we define $TMPDIR as /scratch, which Slurm clea

Re: [slurm-users] Hide Filesystem From Slurm

2019-07-11 Thread Brian Andrus
I don't think that is possible. At least not easily I just symlink /tmp to /scratch on systems I use. That way folks can get used to /scratch, but if anything has hard-coded /tmp, it will still work. Brian Andrus On 7/11/2019 8:19 AM, Douglas Duckworth wrote: Hello I am wondering if it

[slurm-users] Hide Filesystem From Slurm

2019-07-11 Thread Douglas Duckworth
Hello I am wondering if it's possible to hide a file system, that's world writable on compute node, logically within Slurm. That way any job a user runs cannot possible access this file system. Essentially we define $TMPDIR as /scratch, which Slurm cleans up in epilogue scripts, but some user