We tell our users to do this:
squeue -h -t R -O gres | grep gpu|wc -l
The command above will report the number of GPUs in use. If the number is 16,
then all of the GPUs are currently being used. If nothing is displayed, then
all of the GPUs are available.
In our case we have 16 GPU's. Probably better ways then that, but it works.
Best,
Chris
—
Christopher Coffey
High-Performance Computing
Northern Arizona University
928-523-1167
On 3/16/18, 12:19 PM, "slurm-users on behalf of Alex Chekholko"
<slurm-users-boun...@lists.schedmd.com on behalf of a...@calicolabs.com> wrote:
There was a previous thread where someone recommended a third-party script:
"pestat -G" that will parse the outputs of 'scontrol shown node' and 'scontrol
show job' and add up the used GPUs perhaps?
https://github.com/OleHolmNielsen/Slurm_tools/tree/master/pestat
<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOleHolmNielsen%2FSlurm_tools%2Ftree%2Fmaster%2Fpestat&data=02%7C01%7Cchris.coffey%40nau.edu%7C64f9412663a44143553e08d58b72d72c%7C27d49e9f89e14aa099a3d35b57b2ba03%7C0%7C1%7C636568247709507960&sdata=R7wjvsTmFk75ObJrFC7Hif2txBy051xlr%2FPNeE2Dn2Q%3D&reserved=0>
On Fri, Mar 16, 2018 at 11:44 AM, jayraj shah
<jayraj.sha...@gmail.com> wrote:
Thank you.
I tried that option but it gives me, how many gpus are available/attached
to node. (such as gpu:8)
I want to know how many are used and how many are free (to run my jobs on
free gpus).
On Fri, Mar 16, 2018 at 2:34 PM,
sefa.ars...@tubitak.gov.tr <mailto:sefa.ars...@tubitak.gov.tr>
<sefa.ars...@tubitak.gov.tr> wrote:
You msy use sinfo with parameters i.e. %G .
https://slurm.schedmd.com/sinfo.html
<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fslurm.schedmd.com%2Fsinfo.html&data=02%7C01%7Cchris.coffey%40nau.edu%7C64f9412663a44143553e08d58b72d72c%7C27d49e9f89e14aa099a3d35b57b2ba03%7C0%7C1%7C636568247709507960&sdata=7d1Q49XGS3gWed99hMfn9ItT6p6pyWi6PDfW64eGWEw%3D&reserved=0>
Sefa Arslan
------ Orijinal mesaj------
Kimden: jayraj shah
Tarih: Cum, 16 Mar 2018 21:18
Kime: slurm-us...@schedmd.com;
Konu:[slurm-users] Available gpus ?
I am trying to find out how as a user, I can get information about
available/unused gpus on cluster.
This cluster is setup in a way that user can request gpus per node between
1 to 8. Using sinfo it shows that STATE is "mix". But I can't tell how many out
of 8 are being used and how many are available on that mix state node for other
user to
use.
Please let me know your suggestions.
--
Thanks
Jayraj
--
Thanks
Jayraj
