Re: [slurm-users] Cleanup of job_container/tmpfs

2023-03-01 Thread Jason Ellul
Hi Michael, Thanks so much for the info will try 23.02. Cheers, Jason Jason Ellul Head - Research Computing Facility Office of Cancer Research Peter MacCallum Cancer Center From: slurm-users on behalf of Michael Jennings Date: Thursday, 2 March 2023 at 9:17 am To: slurm-users@lists.schedmd

Re: [slurm-users] Cleanup of job_container/tmpfs

2023-03-01 Thread Jason Ellul
Thanks so much Ole for the info and link, Your documentation is extremely useful. Prior to moving to 22.05 we had been using slurm-spank-private-tmpdir with an epilog to clean-up the folders on job completion, but we were hoping to move to the inbuilt functionality to ensure future compatibilit

Re: [slurm-users] Cleanup of job_container/tmpfs

2023-03-01 Thread Michael Jennings
On Wednesday, 01 March 2023, at 10:28:24 (+0100), Ole Holm Nielsen wrote: but there may be some significant improvements included in 23.02 TL;DR: I can vouch for this. The primary problem with the interaction between the new namespace code and the automounter daemon was simply that the shar

[slurm-users] Does Slurm have any equivalent to LSF elim for generating dynamic node resources

2023-03-01 Thread Amir Ben Avi
Hi, I have looked on the Slurm documentation, but didn't found any way to crate resource dynamically ( in a script ) on the node level I also saw that Slurm have the feature + gres that can be declare on the node, but those seems static resource tagging Will also be interesting to know what is t

Re: [slurm-users] srun: Job step aborted

2023-03-01 Thread Niccolo Tosato
I finally solved the issue, my slurm client on computational node was build and configured with pmix_v3 as follow: $:/usr/local/lib/slurm$ ll | grep pmix lrwxrwxrwx 1 root root   16 feb 23 15:57 mpi_pmix.so -> ./mpi_pmix_v3.so* -rwxr-xr-x 1 root root 1003 feb 23 15:57 mpi_pmix_v3.la* -r

Re: [slurm-users] Power saving and node weight

2023-03-01 Thread Brian Andrus
Gizo, There is no documentation, only the bug report mentioned. We may be able to help from the list if you explain what you want to do. Brian On 3/1/2023 12:29 AM, Gizo Nanava wrote: Hello Brian, thanks a lot for the info. You may be able to use the alternate approach that I was able to d

Re: [slurm-users] Cleanup of job_container/tmpfs

2023-03-01 Thread Ole Holm Nielsen
Hi Jason, IMHO, the job_container/tmpfs is not working well in Slurm 22.05, but there may be some significant improvements included in 23.02 (announced yesterday). I've documented our experiences in the Wiki page https://wiki.fysik.dtu.dk/Niflheim_system/Slurm_configuration/#temporary-job-dir

Re: [slurm-users] Power saving and node weight

2023-03-01 Thread Gizo Nanava
Hello Brian, thanks a lot for the info. > > You may be able to use the alternate approach that I was able to do as well. > I would be insterested in any alternatives. Could you point me to some doc? Best wishes Gizo > Brian Andrus > > > On 2/28/2023 7:44 AM, Gizo Nanava wrote: > > Hello, >