[slurm-users] Re: Can't specify multiple partitions when submitting GPU jobs

2025-04-29 Thread milad--- via slurm-users
My partitions definition is super simple: ``` PartitionName=t4 Nodes=slurm-t4-[1-30] DEFAULT=YES MaxTime=INFINITE State=UP DefCpuPerGPU=16 DefMemPerGPU=14350 PartitionName=a100-40 Nodes=slurm-a100-40gb-[1-30] MaxTime=INFINITE State=UP DefCpuPerGPU=12 DefMemPerGPU=85486 PartitionName=a100-80 Node

[slurm-users] Re: Can't specify multiple partitions when submitting GPU jobs

2025-04-28 Thread milad--- via slurm-users
Update: I also noticed that specifying -ntasks makes a difference when --gpus is present. if I have two partitions a100 and h100 that both have free GPUs: ✅ h100 specified first in -p: works sbatch -p h100,a100 --gpus h100:1 script.sh ❌ h100 specified second: doesn't work sbatch -p a100,h100 --

[slurm-users] Can't specify multiple partitions when submitting GPU jobs

2025-04-28 Thread milad--- via slurm-users
Hi, I have defined a partition for each GPU type we have in the cluster. This was mainly because I have different Node types for each GPU type and I want to set `DefCpuPerGPU` `DefMemPerGPU` for each of them. Unfortunately one can't set them per node but can do that per partition. Now sometime