> -j <job id> -l` too. However, it seems to include any GPU index information > even in AllocGres and AllocTres columns.
It DOES NOT seem to include any GPU index, I meant. Sorry. Best. -------------------------------------------- 露崎 浩太 (Kota Tsuyuzaki) [email protected] NTTソフトウェアイノベーションセンタ 分散処理基盤技術プロジェクト 0422-59-2837 --------------------------------------------- > -----Original Message----- > From: slurm-users <[email protected]> On Behalf Of Kota > Tsuyuzaki > Sent: Wednesday, June 10, 2020 11:37 AM > To: 'Slurm User Community List' <[email protected]> > Subject: Re: [slurm-users] How to view GPU indices of the completed jobs? > > > Using sacct you can find those information, try the below options and see > > if that works. > > > > sacct -j <job id> --format=jobid,ReqTRES%50,ReqGres > > Thanks, I tried that command but it looks to show the requested number of > GPUs instead of the GPU index. I tried ` sacct > -j <job id> -l` too. However, it seems to include any GPU index information > even in AllocGres and AllocTres columns. > > Do I have to turn on some configurations to track the detailed GPU > information? Am I missing something? > > Best regards, > > -------------------------------------------- > 露崎 浩太 (Kota Tsuyuzaki) > [email protected] > NTTソフトウェアイノベーションセンタ > 分散処理基盤技術プロジェクト > 0422-59-2837 > --------------------------------------------- > > > > -----Original Message----- > > From: slurm-users <[email protected]> On Behalf Of > > sathish > > Sent: Monday, June 8, 2020 11:07 PM > > To: Slurm User Community List <[email protected]> > > Subject: Re: [slurm-users] How to view GPU indices of the completed jobs? > > > > Using sacct you can find those information, try the below options and see > > if that works. > > > > sacct -j <job id> --format=jobid,ReqTRES%50,ReqGres > > > > > > On Thu, Jun 4, 2020 at 1:30 PM Kota Tsuyuzaki > > <[email protected] <mailto:[email protected]> > > > wrote: > > > > > > Hello Guys, > > > > We are running GPU clusters with Slurm and SlurmDBD (version 19.05 > > series) and some of GPUs seemed to get troubles for attached > > jobs. To investigate if the troubles happened on the same GPUs, I'd > > like to get GPU indices of the completed jobs. > > > > In my understanding `scontrol show job` can show the indices (as IDX > > in gres info) but cannot be used for completed job. And also > > `sacct -j` is available for complete jobs but won't print the indices. > > > > Is there any way (commands, configurations, etc...) to see the > > allocated GPU indices for completed jobs? > > > > Best regards, > > > > -------------------------------------------- > > 露崎 浩太 (Kota Tsuyuzaki) > > [email protected] <mailto:[email protected]> > > NTTソフトウェアイノベーションセンタ > > 分散処理基盤技術プロジェクト > > 0422-59-2837 > > --------------------------------------------- > > > > > > > > > > > > > > > > > > > > -- > > > > Regards..... > > Sathish >
