We run in that manner using this config on v.3.10.0-693.5.2.el7.x86_64 This is slurm 17.02.4
Do your compute nodes have hyperthreading enabled ? AuthType=auth/munge CryptoType=crypto/munge AccountingStorageEnforce=limits,qos,safe AccountingStoragePort=ANumber AccountingStorageType=accounting_storage/slurmdbd AccountingStoreJobComment='yes' AccountingStorageUser=slurm CacheGroups=0 EnforcePartLimits='yes' FastSchedule=1 GresTypes=gpu InactiveLimit=0 JobAcctGatherFrequency=30 JobAcctGatherType=jobacct_gather/cgroup KillWait=30 Licenses=mplus:1 MaxArraySize=40001 MaxJobCount=350000 MinJobAge=300 MpiDefault=none PriorityDecayHalfLife=14-0 PriorityFavorSmall='no' PriorityFlags=fair_tree PriorityMaxAge=60-0 PriorityType=priority/multifactor PriorityWeightAge=1000 PriorityWeightFairshare=10000 PriorityWeightJobSize=1000 PriorityWeightPartition=1000 PriorityWeightQOS=1000 ProctrackType=proctrack/cgroup RebootProgram=/usr/sbin/reboot ReturnToService=2 SallocDefaultCommand='"srun -n1 -N1 --gres=gpu:0 --mem-per-cpu=0 --pty --preserve-env --mpi=none $SHELL"' SchedulerPort=ANumber SchedulerParameters=kill_invalid_depend SchedulerType=sched/backfill SelectTypeParameters=CR_CPU_Memory SelectType=select/cons_res SlurmctldDebug=3 SlurmctldPort=NumberRange SlurmctldTimeout=120 SlurmdDebug=3 SlurmdPort=ANumber SlurmdTimeout=300 SlurmUser=slurm SwitchType=switch/none TaskPlugin=task/cgroup Waittime=0 ANumber are port numbers or ranges. From: slurm-users [mailto:slurm-users-boun...@lists.schedmd.com] On Behalf Of Ethan Van Matre Sent: Tuesday, November 28, 2017 7:32 PM To: slurm-users@lists.schedmd.com Subject: [slurm-users] fail when trying to set up selection=con_res I've been trying to setup a slurm cluster with con_res enabled. No luck. Running on ubuntu 16.04 When using linear selection all works as expected. Jobs are schedules and run their course then exit. Exclusive use of the node is granted. We would like to schedule based on cpu (cores actually) and set thus: # SCHEDULING FastSchedule=1 SchedulerType=sched/backfill #SchedulerPort=7321 #SelectType=select/linear SelectType=select/cons_res SelectTypeParameters=CR_CORE #SelectTypeParameters=CR_CPU When launching jobs we more than one at a time per node but the jobs become hung in a COMPLETING state. Not sure if they ever started. Can anyone point me to how to set up slurm so that allocation is on a cpu (core) basis with as many jobs as there are cores running on each node? Regards Ethan VanMatre Informatics Research Analyst Institute on Development and Disability Oregon Health & Science University CSLU - GH40 3181 SW Sam Jackson Park Rd Portland, OR 97239 (503) 346-3764 vanma...@ohsu.edu<mailto:vanma...@ohsu.edu>
