I think this would be a good feature request. At least to me everything
you can get in scontrol show job should be in sacct in some form.
-Paul Edmon-
On 8/7/2024 9:29 AM, Steffen Grunewald wrote:
On Wed, 2024-08-07 at 08:55:21 -0400, Slurm users wrote:
Warning on that one, it can eat up a to
Maybe a somewhat 'hacky' idea - couldn't you put a line in the epilog
script that logs the interesting entries (i.e. SLURM_SUBMIT_HOST) to
some logfile at job completion? Of course that would only be feasible if
the amount of completing jobs per time unit isn't super high, otherwise
you'd obvio
Hi Paul,
agreed. Actually, we even have
AccountingStoreFlags=job_comment,job_script,job_env
set in slurm.conf. Although we didn't observe any issues so far in
regular operation with about 1 jobs per day, this did clearly
affect database conversion time during major Slurm version updates.
If you need it, you could add it to either prologue or epilogue to store
the info somewhere.
I do that for the scripts themselves and keep the past two weeks backed
up so we can debug if/when there is an issue.
Brian Andrus
On 8/7/2024 6:29 AM, Steffen Grunewald via slurm-users wrote:
On We
On Wed, 2024-08-07 at 08:55:21 -0400, Slurm users wrote:
> Warning on that one, it can eat up a ton of database space (depending on
> size of environment, uniqueness of environment between jobs, and number of
> jobs). We had it on and it nearly ran us out of space on our database host.
> That said
Hello Steffen,
On 8/7/24 2:08 PM, Steffen Grunewald via slurm-users wrote:
Is there a way to find out what the value of SLURM_SUBMIT_HOST was?
I'd be interested in a few more env variables, but this one should be
sufficient for a start...
What you're looking for might be doable simply by setti
Warning on that one, it can eat up a ton of database space (depending on
size of environment, uniqueness of environment between jobs, and number
of jobs). We had it on and it nearly ran us out of space on our database
host. That said the data can be really useful depending on the situation.
-P
Hi Steffen,
not sure if this is what you are looking for, but with
`AccountingStoreFlags=job_env´
set in slurm.conf, the batch job environment will be stored in the
accounting database and can later be retrieved with `sacct -j
--env-vars´
command.
We find this quite useful for debugging purp
That looks to be the case from my glance at sacct. Not everything in
scontrol show job ends up in sacct, which is a bit frustrating at times.
-Paul Edmon-
On 8/7/2024 8:08 AM, Steffen Grunewald via slurm-users wrote:
Hello everyone,
I've grepped the manual pages and crawled the 'net, but coul
Hello everyone,
I've grepped the manual pages and crawled the 'net, but couldn't find any
answer to the following problem:
I can see that the ctld keeps a record of it below /var/spool/slurm - as
long as the job is running or waiting (and shown by "squeue") - and that
this stores the environment
10 matches
Mail list logo
