Re: [slurm-users] Get Job Array information in Epilog script

2023-03-17 Thread Timo Rothenpieler
On 17/03/2023 13:11, William Brown wrote: We create the temporary directories using SLURM_JOB_ID, and that works fine with Job Arrays so far as I can see. Don't you have a problem if a user has multiple jobs on the same node? William Ours users just have /work/$username, anything below that

Re: [slurm-users] Get Job Array information in Epilog script

2023-03-17 Thread William Brown
We create the temporary directories using SLURM_JOB_ID, and that works fine with Job Arrays so far as I can see. Don't you have a problem if a user has multiple jobs on the same node? William On Fri, 17 Mar 2023 at 11:17, Timo Rothenpieler wrote: > > Hello! > > I'm currently facing a bit of an

[slurm-users] Get Job Array information in Epilog script

2023-03-17 Thread Timo Rothenpieler
Hello! I'm currently facing a bit of an issue regarding cleanup after a job completed. I've added the following bit of Shellscript to our clusters Epilog script: for d in "${SLURM_JOB_ID}" "${SLURM_JOB_ID}_${SLURM_ARRAY_TASK_ID}" "${SLURM_ARRAY_JOB_ID}_${SLURM_ARRAY_TASK_ID}"; do WO

[slurm-users] Get required memory from jcomp_elasticsearch.c

2022-02-21 Thread Elisabeth Ortega
Hi there, we're using the jcomp_elasticsearch.c plugin ( https://github.com/SchedMD/slurm/blob/master/src/plugins/jobcomp/elasticsearch/jobcomp_elasticsearch.c) to get some data for monitoring but we have to use the slurm-perlapi to gather the information related to the required memory due to the

[slurm-users] Get Information from a Node to the MailProg Command / Add arbitrary information to a job

2021-06-15 Thread Matthias Loose
Hi Slurm Users, first time posting. I have a new slurm setup where the users can specify an amount of local node disk space they wish to use. This is a "gres" resource named "local" and it measures in GB. Once the user has scheduled a job and it gets executed, I create a folder for this job on

[slurm-users] get job status of completed & cleared jobs from rest interface

2021-05-28 Thread Simone Riggi
Dear all, I am writing to ask you a question. Is it possible to retrieve the status of cleared jobs (e.g. after completed with either success or failed) from the Slurm rest interface ? When a job (job id=131 in the example below) is cleared, the rest interface returns this after some time after c

Re: [slurm-users] Get original script of a job

2021-03-07 Thread Alberto Morillas, Angelines
Message: 1 Date: Fri, 5 Mar 2021 11:56:05 +0100 From: Ole Holm Nielsen To: Subject: Re: [slurm-users] Get original script of a job Message-ID: <61c47956-5fc9-5be5-9aef-08f8e27bf...@fysik.dtu.dk> Content-Type: text/plain; charset="utf

Re: [slurm-users] Get original script of a job

2021-03-05 Thread Carl Ponder
script with each of the runs. So I can put it in the same directory where all the other output files go, and have a complete description of the run conditions. ---- Subject: [slurm-users] Get original script of a job Dat

Re: [slurm-users] Get original script of a job

2021-03-05 Thread Ward Poelmans
Hi, On 5/03/2021 11:29, Alberto Morillas, Angelines wrote: > I know that when I send a job with scontroI can get the path and the > name of the script used to send this job, but normally the users change > theirs scripts and sometimes all was wrong after that, so is there any > possibility to rep

Re: [slurm-users] Get original script of a job

2021-03-05 Thread Ole Holm Nielsen
On 05-03-2021 11:29, Alberto Morillas, Angelines wrote: I would like to know if it will be possible to get the script that was used to send a job. I know that when I send a job with scontroI can get the path and the name of the script used to send this job, but normally the users change their

[slurm-users] Get original script of a job

2021-03-05 Thread Alberto Morillas, Angelines
Hi, I would like to know if it will be possible to get the script that was used to send a job. I know that when I send a job with scontroI can get the path and the name of the script used to send this job, but normally the users change theirs scripts and sometimes all was wrong after that, so i

Re: [slurm-users] get

2019-12-16 Thread Wiegand, Paul
Okay ... obviously an auto-complete error that I failed to check: Please ignore and accept my apologies. > On Dec 16, 2019, at 7:03 AM, Wiegand, Paul wrote: > > unlock stokes-arcc > get stokes-arcc >

[slurm-users] get

2019-12-16 Thread Wiegand, Paul
unlock stokes-arcc get stokes-arcc

Re: [slurm-users] Get GPU usage from sacct?

2019-11-19 Thread Janne Blomqvist
On 15/11/2019 17.06, Miguel Oliveira wrote: Thank! Nice code and just what I was needing! A few wrinkles: a) on reading the Gres from scontrol for each job on my version this is on a TRES record not as an individual Gres. Possibly version/configuration issue. b) converting pid2id from /proc//cg

Re: [slurm-users] Get GPU usage from sacct?

2019-11-16 Thread Aaron Jackson
Janne Blomqvist writes: > On 14/11/2019 20.41, Prentice Bisbal wrote: >> Is there any way to see how much a job used the GPU(s) on a cluster >> using sacct or any other slurm command? >> > > We have created > https://github.com/AaltoScienceIT/ansible-role-sacct_gpu/ as a quick > hack to put GPU uti

Re: [slurm-users] Get GPU usage from sacct?

2019-11-15 Thread Miguel Oliveira
Thank! Nice code and just what I was needing! A few wrinkles: a) on reading the Gres from scontrol for each job on my version this is on a TRES record not as an individual Gres. Possibly version/configuration issue. b) converting pid2id from /proc//cgroup is problematic on array jobs. Again many

Re: [slurm-users] Get GPU usage from sacct?

2019-11-15 Thread Janne Blomqvist
On 14/11/2019 20.41, Prentice Bisbal wrote: > Is there any way to see how much a job used the GPU(s) on a cluster > using sacct or any other slurm command? > We have created https://github.com/AaltoScienceIT/ansible-role-sacct_gpu/ as a quick hack to put GPU utilization stats into the comment fie

Re: [slurm-users] Get GPU usage from sacct?

2019-11-14 Thread Ryan Novosielski
Do you mean akin to what some would consider "CPU efficiency" on a CPU job? "How much... used" is a little vague. From: slurm-users on behalf of Prentice Bisbal Sent: Thursday, November 14, 2019 13:41 To: Slurm User Community List S

[slurm-users] Get GPU usage from sacct?

2019-11-14 Thread Prentice Bisbal
Is there any way to see how much a job used the GPU(s) on a cluster using sacct or any other slurm command? -- Prentice

Re: [slurm-users] Get list of nodes and their status, one node per line, no duplicates

2017-11-08 Thread Kilian Cavalotti
Hi Jeff, Quite close: $ sinfo --Format=nodehost,statelong Cheers, -- Kilian

Re: [slurm-users] Get list of nodes and their status, one node per line, no duplicates

2017-11-08 Thread Lachlan Musicman
I use alias sn='sinfo -Nle -o "%.20n %.15C %.8O %.7t" | uniq' and then it's just [root@machine]# sn cheers L. -- "The antidote to apocalypticism is *apocalyptic civics*. Apocalyptic civics is the insistence that we cannot ignore the truth, nor should we panic about it. It is a shared consc

[slurm-users] Get list of nodes and their status, one node per line, no duplicates

2017-11-08 Thread Jeff White
Subject says it all.  Is there a way to get a list of nodes, their status, and NOT have duplicate entries in the output?  This is what I have so far but it seems to duplicate nodes if they exist in more than 1 partition, which is true of all my nodes. sinfo --Node --Format=nodelist,statelong