Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Jeherul Islam
Hi Jürgen, No reservation was done during that time. On Wed, Sep 1, 2021 at 7:11 PM Juergen Salk wrote: > Dear Jeherul, > > sacct is for job accounting, sreport for cluster usage accounting. > > Did you maybe had any resource reservations for this user in place during > that period of time? > >

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Jeherul Islam
Hi Loris, No output is truncated. Here is the snapshot of the output. #sacct -X --account=chemistry --user=j.mira --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31 --noheader 23269 j.mira1209627TIMEOUTgpu:1 1 25853

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Paul Raines
Doesn't add up for me either # sacct -X -P --format=jobid,partition,ElapsedRaw,state,AllocTRES --starttime=2021-08-01 --endtime=2021-08-31 --user=jl1103 | grep gpu= 470069|1|FAILED|billing=4,cpu=1,gres/gpu=1,mem=20G,node=1 470351|1|FAILED|billing=4,cpu=1,gres/gpu=1,mem=20G,node=1 470352|1|F

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Juergen Salk
Dear Jeherul, sacct is for job accounting, sreport for cluster usage accounting. Did you maybe had any resource reservations for this user in place during that period of time? To my very best knowledge, resource reservations for one or more users do count in terms of cluster usage as reporte

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Tina Friedrich
...or maybe sacct -p --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus i.e. make the output parsable (non-truncated and delimiter separated) - handy if you want/need to do further work with the data. Tina On 01/09/2021 14:24, Loris Bennett wrote: Dear Jeherul, Jeherul Islam writes: D

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Loris Bennett
Dear Jeherul, Jeherul Islam writes: > Dear Loris, > > When we grep it by the user name "j.mira" it will strike out the multiple > counts. Again sacct is showing fewer gpu minutes than sreport. Yes, you are right, although instead of sacct --account=chemistry --format=jobid,user,ElapsedR

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Jeherul Islam
Dear Loris, When we grep it by the user name "j.mira" it will strike out the multiple counts. Again sacct is showing fewer gpu minutes than sreport. On Wed, 1 Sep, 2021, 6:03 PM Loris Bennett, wrote: > Dear Jeherul, > > Jeherul Islam writes: > > > Dear Loris, > > > > Thanks for your reply. Her

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Loris Bennett
Dear Jeherul, Jeherul Islam writes: > Dear Loris, > > Thanks for your reply. Here is the output for the same period but the result > is not matching. > > #sacct --account=chemistry > --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 > --endtime=2021-08-31 | grep j.

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Jeherul Islam
Dear Loris, Thanks for your reply. Here is the output for the same period but the result is not matching. #sacct --account=chemistry --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31 | grep j.mira| grep gpu| awk '{sum += $3} END {print sum}' *68350

Re: [slurm-users] Calculate the GPU usages

2021-09-01 Thread Loris Bennett
Dear Jeherul, Jeherul Islam writes: > Dear All, > > Please share the correct way of calculating the GPU usages. > I am confused with sreport and sacct cmd. I am getting a different result. > > # sreport cluster AccountUtilizationByUser cluster= user=j.mira > start=2021-05-01 end=2021-08-3

[slurm-users] Calculate the GPU usages

2021-09-01 Thread Jeherul Islam
Dear All, Please share the correct way of calculating the GPU usages. I am confused with sreport and sacct cmd. I am getting a different result. # sreport cluster AccountUtilizationByUser cluster= user=j.mira start=2021-05-01 end=2021-08-31 --tres="gres/gpu" --