Re: [slurm-users] Interactive jobs using "srun --pty bash" and MPI

2022-11-02 Thread Christopher Samuel
On 11/2/22 4:45 pm, Juergen Salk wrote: However, instead of using `srun --pty bash´ for launching interactive jobs, it is now recommended to use `salloc´ and have `LaunchParameters=use_interactive_step´ set in slurm.conf. +1 on that, this is what we've been using since it landed. -- Chris Sa

Re: [slurm-users] Interactive jobs using "srun --pty bash" and MPI

2022-11-02 Thread Juergen Salk
Hi Em, this is most probably because in Slurm version 20.11 the behaviour of srun was changed to not allow job steps to overlap by default any more. An interactive job launched by `srun --pty bash´ always creates a regular step (step .0), so mpirun or srun will hang when trying to launch anoth

[slurm-users] Interactive jobs using "srun --pty bash" and MPI

2022-11-02 Thread Em Dragowsky
Greetings -- When we started using Slurm some years ago, obtaining the interactive resources through "srun ... --pty bash" was the standard that we adopted. We are now running Slurm v22.05 (happily), though we noticed recently some limitations when claiming resources to demonstrate or develop in a

Re: [slurm-users] SlurmDBD losing connection to the backend MariaDB

2022-11-02 Thread Richard Chang
Hello Brian, Thank you for the reply and sharing your design. Can you please share your MariaDB server HA details.? ( Can be offline and DM to me ) I would like to understand it so that I can replicate it  here. Thanks & regards, Richard. On 11/2/2022 8:09 AM, Brian Andrus wrote: RC, In t

Re: [slurm-users] slurm accounting shows more MaxRSS than physically available memory

2022-11-02 Thread Ohlerich, Martin
Dear Jürgen, man thanks for your reply, and your thoughts! What you say makes deductively much sense to me ;) It is only confusing in respect to the SchedMD documentation, as I pointed out. (MaxRSSTask is then distinguished to be what?) So, if you are right, this includes really everything,

Re: [slurm-users] slurm accounting shows more MaxRSS than physically available memory

2022-11-02 Thread Juergen Salk
Hi Martin, to my very best knowledge MaxRSS does report aggregated memory consumption of all tasks but including all the shared libraries that the individual processes uses, even though a shared library is only loaded into memory once regardless of how many processes use it. So shared librarie

[slurm-users] slurm accounting shows more MaxRSS than physically available memory

2022-11-02 Thread Ohlerich, Martin
Dear "Commiserates". I wonder a bit about the meaning of MaxRSS. The documentation says: "Maximum resident set size of all tasks in job." To what refers here "maximum"? The maximum over job period, I understand hopefully correctly. But it does not seem to be the size of all tasks (summed up, so-