[slurm-users] Re: Job running slower when using Slurm

2025-04-24 Thread Jeffrey Layton via slurm-users
e can support you > by visiting our Service Catalogue > <https://azcollaboration.sharepoint.com/sites/CMU993> | > > > > > > *From: *Michael DiDomenico via slurm-users > *Date: *Wednesday, 23 April 2025 at 7:53 pm > *To: * > *Cc: *Slurm User Community List

[slurm-users] Re: Job running slower when using Slurm

2025-04-24 Thread Cutts, Tim via slurm-users
n support you by visiting our Service Catalogue<https://azcollaboration.sharepoint.com/sites/CMU993> | From: Michael DiDomenico via slurm-users Date: Wednesday, 23 April 2025 at 7:53 pm To: Cc: Slurm User Community List Subject: [slurm-users] Re: Job running slower when using Slurm the

[slurm-users] Re: Job running slower when using Slurm

2025-04-24 Thread Jeffrey Layton via slurm-users
Thanks for the suggestion but I already do that. Jeff On Wed, Apr 23, 2025 at 4:45 PM Feng Zhang wrote: > Beside slurm options, you might also need to set OpenMP env variable: > > export OMP_NUM_THREADS=32 (the core, not thread number) > > Also other similar env variables, if you use any Pytho

[slurm-users] Re: Job running slower when using Slurm

2025-04-23 Thread Feng Zhang via slurm-users
Beside slurm options, you might also need to set OpenMP env variable: export OMP_NUM_THREADS=32 (the core, not thread number) Also other similar env variables, if you use any Python libs. Best, Feng On Wed, Apr 23, 2025 at 3:22 PM Jeffrey Layton via slurm-users < slurm-users@lists.schedmd.com

[slurm-users] Re: Job running slower when using Slurm

2025-04-23 Thread Jeffrey Layton via slurm-users
Roger. It's the code that prints out the threads it sees - I bet it is the cgroups. I need to look at how that it is configured as well. For the time, that comes from the code itself. I'm guessing it has a start time and and end time in the code and just takes the difference. But again, this is so

[slurm-users] Re: Job running slower when using Slurm

2025-04-23 Thread Michael DiDomenico via slurm-users
the program probably says 32 threads, because it's just looking at the box, not what slurm cgroups allow (assuming your using them) for cpu i think for an openmp program (not openmpi) you definitely want the first command with --cpus-per-task=32 are you measuring the runtime inside the program or

[slurm-users] Re: Job running slower when using Slurm

2025-04-23 Thread Jeffrey Layton via slurm-users
I tried using ntasks and cpus-per-task to get all 32 cores. So I added --ntasks=# --cpus-per-task=N to th sbatch command so that it now looks like: sbatch --nodes=1 --ntasks=1 --cpus-per-task=32

[slurm-users] Re: Job running slower when using Slurm

2025-04-23 Thread Jeffrey Layton via slurm-users
Roger. I didn't configure Slurm so let me look at slurm.conf and gres.conf to see if they restrict a job to a single CPU. Thanks On Wed, Apr 23, 2025 at 1:48 PM Michael DiDomenico via slurm-users < slurm-users@lists.schedmd.com> wrote: > without knowing anything about your environment, its reaso

[slurm-users] Re: Job running slower when using Slurm

2025-04-23 Thread Michael DiDomenico via slurm-users
without knowing anything about your environment, its reasonable to suspect that maybe your openmp program is multi-threaded, but slurm is constraining your job to a single core. evidence of this should show up when running top on the node, watching the cpu% used for the program On Wed, Apr 23, 20