Re: [slurm-users] Performance tracking of array tasks

2022-05-17 Thread William Dear
e end up with high IO wait so identifying the culprit requires also comparing total IO per job. William Dear From: slurm-users on behalf of Loris Bennett Sent: Tuesday, May 17, 2022 12:46 AM To: Slurm User Community List Subject: Re: [slurm-users] Performanc

Re: [slurm-users] Performance tracking of array tasks

2022-05-16 Thread William Dear
ince it only runs 100 at a time but all the pending array jobs still show up as waiting. If the partition resources are too low and the job is running less than 100 then it actually is waiting on another job. The challenge will be determining when a job is self limiting vs waiting on

[slurm-users] Performance tracking of array tasks

2022-05-16 Thread William Dear
the job is complete I would like to see run time and peak RAM usage per task so that we can correctly size the reservations for future jobs. It would also be very helpful to break this down by node so that I can identify poorly performing nodes. William Dear

[slurm-users] Phantom jobs in sreport

2022-05-10 Thread William Dear
I have several phantom jobs in "sreport user" that always show up even during periods when no jobs were running. I don’t know the job IDs and would appreciate suggestions for identifying the jobs and fixing the database. My cluster accounting is uses slurmdbd with mariadb. sreport user TopU