[slurm-users] Job are pending when plenty of resources available

I'm perplexed. My cluster has been churning along and tonight it has decided to start pending jobs even though there are plenty of nodes available. An example job from squeue: JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) 409978 interactiver

Re: [slurm-users] Is it safe to convert cons_res to cons_tres on a running system?

Sounds pretty safe, but with the current COVID-19 difficulties, including Spring quarter classes being taught remotely (starts tomorrow, fun times ahead), I'm a bit reluctant to poke a running system. This will get put on the giant list of things waiting for a scheduled downtime once campus re-open