On 1/29/21 3:51 AM, taleinterve...@sjtu.edu.cn wrote:
Thanks for the help. The doc page is useful and we can get the actual job id
now.
I'm glad that you solved the problem.
The reason we need to delete job record from database is our billing system
will calculate user cost from these historical records. But after a slurm
system faulty there will be some specific jobs which should not be charged. it
seems the best practical solution is to directly modify the database since
slurm does not provide commend to delete job records.
I understand, but I think there ought to be a way to set the charging cost
(money) of specific jobs in the database to zero. I will post a separate
message about that.
/Ole
-----邮件原件----->
--
Ole Holm Nielsen
PhD, Senior HPC Officer
Department of Physics, Technical University of Denmark,
Fysikvej Building 309, DK-2800 Kongens Lyngby, Denmark
E-mail: ole.h.niel...@fysik.dtu.dk
Homepage: http://dcwww.fysik.dtu.dk/~ohnielse/
Mobile: (+45) 5180 1620
发件人: Ole Holm Nielsen <ole.h.niel...@fysik.dtu.dk>
发送时间: 2021年1月29日 0:14
收件人: slurm-users@lists.schedmd.com
主题: Re: [slurm-users] how do array jobs stored in slurmdb database?
On 1/28/21 11:59 AM, taleinterve...@sjtu.edu.cn wrote:
From query command such as ‘sacct -j 123456’ I can see a series of
jobs named 123456_1, 123456_2, etc. And I need to delete these job
records from mysql database for some reason.
But in job_table of slurmdb, there is only one record with id_job=123456.
not any record has a id like 123456_2. After I delete the
id_job=123456 record, sacct result show the 123456_1 job disappeared,
but other jobs in the array still exist. So how do these array job recorded in
the database?
And how to completely delete all the jobs in a array?
I think you need to study how job arrays are implemented in Slurm, please read
https://slurm.schedmd.com/job_array.html
You will discover that job arrays, when each individual jobs start running,
become independent jobs and obtain their own unique JobIDs. It must be those
JobIDs that will appear in the Slurm database.
This command illustrates the different JobID types (please read the squeue
manual page about ArrayJobID,JobArrayID,JobID):
$ squeue -j 3394902 -O ArrayJobID,JobArrayID,JobID
ARRAY_JOB_ID JOBID JOBID
3394902 3394902_[18-91] 3394902
3394902 3394902_17 3394919
3394902 3394902_16 3394918
3394902 3394902_15 3394917
3394902 3394902_14 3394916
The last 4 jobs are running, while the first job i still pending.
Perhaps you may find my "showjob" script useful:
https://github.com/OleHolmNielsen/Slurm_tools/tree/master/jobs
In this script you can see how I work with array jobs.
I did not answer your question about how to delete array jobs in the Slurm
database. But in most cases manipulating the database directly is probably a
bad idea. I wonder why you want to delete jobs in the database at all?
Best regards,
Ole
