[slurm-users] Drain a single user's jobs

[Mark Dixon]

Hi all,

I'm a slurm newbie who has inherited a working slurm 16.05.10 cluster.
I'd like to stop user foo from submitting new jobs but allow their 
existing jobs to run.

We have several partitions, each with its own qos and MaxSubmitJobs 
typically set to some vaue. These qos are stopping a "sacctmgr update user 
foo set maxsubmitjobs=0" from doing anything useful, as per the 
documentation.

I've tried setting up a competing qos:

  sacctmgr add qos drain
  sacctmgr modify qos drain set MaxSubmitJobs=0
  sacctmgr modify qos drain set flags=OverPartQOS
  sacctmgr modify user foo set qos=drain

This has successfully prevented the user from submitting new jobs, but 
their existing jobs aren't running. I'm seeing the reason code 
"InvalidQOS".

Any ideas what I should be looking at, please?

Thanks,

Mark