Modify it and raise the priority to something very, very high.
scontrol update job=JOBID priority=9999999
Brian Andrus
On 9/16/2021 8:39 AM, 顏文 wrote:
Dear users
Thank for the immediate replies.I currently have one important job
running. How to prevent the running job from being preempted? That job
has been running for two weeks, and I am afraid it will be terminated
in the next few days. Can we do anything to secure the job from being
cancelled and preempted? Thanks.
regards,
Peter
Russell Jones <arjone...@gmail.com <mailto:arjone...@gmail.com>> 於
2021年9月14日 週二 下午10:52寫道：
The other option is creating a "special" partition that only this
user(s) can submit to, where jobs running in that partition have a
higher priority than all the others (if you are using partition
priority like we are).
On Tue, Sep 14, 2021 at 3:26 AM Loris Bennett
<loris.benn...@fu-berlin.de <mailto:loris.benn...@fu-berlin.de>>
wrote:
Dear Peter,
顏文 <peteryen2...@gmail.com <mailto:peteryen2...@gmail.com>>
writes:
> Dear Mr. Zillner
>
> I would like the specific running job not being rescheduled
, but also can not be terminated or cancelled in any way. If
the job is cancelled, I need to start it over again. Normally
this kind of jobs require weeks to
> finish. So the time costs it take to restart is quite
significant. Can your suggestions also achieve that? Thanks.
I have not tried this, but could you define a QOS which cannot be
preempted by the other QOSs? That way you can also restrict
access to
the QOS, since a non-preemptable QOS will probably be more
attractive
than a preemptable one. You would then obviously have to
explain to the
users which QOS they should use.
Cheers,
Loris
> with regards,
> Peter
>
> Florian Zillner <fzill...@lenovo.com
<mailto:fzill...@lenovo.com>> 於 2021年9月14日 週二 下午3:40寫道：
>
> See the no-requeue option for SBATCH:
>
> --no-requeue
> Specifies that the batch job should never be requeued under
any circumstances. Setting this option will prevent system
administrators from being able to restart the job (for
example, after a scheduled
> downtime), recover from a node failure, or be requeued upon
preemption by a higher priority job. When a job is requeued,
the batch script is initiated from its beginning. Also see the
--requeue option. The
> JobRequeue configuration parameter controls the default
behavior on the cluster.
>
> https://slurm.schedmd.com/sbatch.html
<https://slurm.schedmd.com/sbatch.html>
>
> Get Outlook for Android
>
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> From: slurm-users <slurm-users-boun...@lists.schedmd.com
<mailto:slurm-users-boun...@lists.schedmd.com>> on behalf of
顏文 <peteryen2...@gmail.com <mailto:peteryen2...@gmail.com>>
> Sent: Tuesday, September 14, 2021 7:02:13 AM
> To: slurm-users@lists.schedmd.com
<mailto:slurm-users@lists.schedmd.com>
<slurm-users@lists.schedmd.com
<mailto:slurm-users@lists.schedmd.com>>
> Subject: [External] [slurm-users] How can I do to prevent a
specific job from being prempted?
>
> Dear slurm users,
>
> I have some specific jobs that can't be terminated,
otherwise they need to be rerun from the beginning. Can we
simply apply some settings (either by user or administrator)
so that these jobs will not be
> preempted ? Thanks.
>
> with regards,
> Peter
