Hi, I recently upgraded Slurm from 19.05.5 to 20.02.5 and I noticed that I am no longer able to use the scontrol update command on GRES unless the new count is 0.
For example, if I have a GRES named "test" defined and in slurm.conf > NodeName=node1 Gres=test:2 Initially, I have: $ sinfo -n node1 -O gres -h $ test:2 After scontrol update, count remains the same: $ sudo scontrol update NodeName=node1 Gres=test:1 $ sinfo -n node1 -O gres -h $ test:2 Only updating to 0 seems to be working $ sudo scontrol update NodeName=node1 Gres=test:0 $ sinfo -n node1 -O gres -h $ It does not show any error or output but nothing is updated. Non-zero Gres update was working for me on Slurm 19.05.5 and I did not see anything related in the release note. I only saw that both 19.05.5 <https://slurm.schedmd.com/archive/slurm-19.05.5/scontrol.html> and 20.02.5 <https://slurm.schedmd.com/scontrol.html> scontrol documentation mentions this: > Modification of GRES count associated with specific files (e.g. GPUs) is > not allowed other than to set their count on a node to zero. However, in my slurm.conf, Gres "test" is not specified with a file. Is this the expected behavior of Slurm 20.02.5? Is there any possible workaround (maybe explicitly clear out the 'file' field)? Any help or comments would be greatly appreciated! Best, Chen
