Hi Lorenzo, On 7/20/23 12:16, Lorenzo Bosio wrote: > One more thing I'd like to point out, is that I need to monitor jobs going > from pending to running state (after waiting in the jobs queue). I > currently have a separate pthread to achieve this, but I think at this > point the job_submit()/job_modify() function has already exited. > I do get the output of the slurm_kill_job() function when called, but > that's not useful for the user and I couldn't find a way to append custom > messages.
Maybe it's useful to have E-mail notifications sent to your users when the job transitions its state? According to the sbatch man-page the user can specify himself what mail alerts he would like: --mail-type=<type> Notify user by email when certain event types occur. Valid type values are NONE, BEGIN, END, FAIL, REQUEUE, ALL (equivalent to BEGIN, END, FAIL, INVALID_DEPEND, REQUEUE, and STAGE_OUT), INVALID_DEPEND (dependency never satisfied), STAGE_OUT (burst buffer stage out and teardown completed), TIME_LIMIT, TIME_LIMIT_90 (reached 90 percent of time limit), TIME_LIMIT_80 (reached 80 percent of time limit), TIME_LIMIT_50 (reached 50 percent of time limit) and ARRAY_TASKS (send emails for each array task). /Ole
