[slurm-users] GPU Jobs with Slurm

2021-01-13 Thread Abhiram Chintangal
Hello, I recently set up a small cluster at work using Warewulf/Slurm. Currently, I am not able to get the scheduler to work well with GPU's (Gres). While slurm is able to filter by GPU type, it allocates all the GPU's on the node. See below: [abhiram@whale ~]$ srun --gres=gpu:p100:2 -n 1 --part

[slurm-users] Questions regarding slurm scheduler plugin

2021-01-13 Thread Kaiyue Wu
Hi, We want to modify a job's priority right after it is submitted automatically in slurm. Job submit plugin was considered, but it seems it haven't even had the jobid of a new job yet (probably because the job have't actually been submitted yet), while we need to use scontrol with jobid to get

Re: [slurm-users] [EXTERNAL] Possible to copy sacctmgr info from one cluster to another?

2021-01-13 Thread Julius, Chad
Thanks for the replies, I can dump the file and it has all the data I need. I really appreciate the help. Chad Chad Julius Cyberinfrastructure Engineer Specialist South Dakota State University Old Horticulture 205 Brookings, SD 57007 Phone: 605.688.5767 Email: chad.jul...@sdstate.edu

Re: [slurm-users] [EXTERNAL] Possible to copy sacctmgr info from one cluster to another?

2021-01-13 Thread Tina Friedrich
You can also run both clusters from the same database (with or without federating them). Might not be what you want - you can then e.g. in sinfo see both clusters, and/or submit to all clusters from everywhere and such. Tina On 13/01/2021 18:19, Mando Rodriguez wrote: Hi chad, You can ‘dum

Re: [slurm-users] [EXTERNAL] Possible to copy sacctmgr info from one cluster to another?

2021-01-13 Thread Mando Rodriguez
Hi chad, You can ‘dump’ the info in the slurm database to a file and reload the file (here named cluster.cfg). Dump the info with: sacctmgr dump slurm_cluster file=cluster.cfg You can load the info with: sacctmgr load file=cluster.cfg It saves all accounts, associations, and partition permis

[slurm-users] Possible to copy sacctmgr info from one cluster to another?

2021-01-13 Thread Julius, Chad
All, Curious if it is possible to copy my sacctmgr information (accounts, users, etc...) from one cluster to another? For example slurm_cluster1 to slurm_cluster2. Thanks, Chad Chad Julius Cyberinfrastructure Engineer Specialist South Dakota State University Old Horticulture 205 Brookings, S