Re: [slurm-users] slurmd and dynamic nodes

2022-09-23 Thread Groner, Rob
: Friday, September 23, 2022 12:24 PM To: slurm-users@lists.schedmd.com Subject: Re: [slurm-users] slurmd and dynamic nodes You don't often get email from toomuc...@gmail.com. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> You shouldn't have to change a

Re: [slurm-users] slurmd and dynamic nodes

2022-09-23 Thread Brian Andrus
ynamic node.  What is the preferred method? Rob *From:* slurm-users on behalf of Brian Andrus *Sent:* Friday, September 23, 2022 10:24 AM *To:* slurm-users@lists.schedmd.com *Subject:* Re: [slurm-users] slurmd and dyna

Re: [slurm-users] slurmd and dynamic nodes

2022-09-23 Thread Groner, Rob
u handle slurmd running on the dynamic node. What is the preferred method? Rob From: slurm-users on behalf of Brian Andrus Sent: Friday, September 23, 2022 10:24 AM To: slurm-users@lists.schedmd.com Subject: Re: [slurm-users] slurmd and dynamic nodes You don&#x

Re: [slurm-users] slurmd and dynamic nodes

2022-09-23 Thread Brian Andrus
Just off the top of my head here. I would expect you need to have no jobs currently running on the node, so you could could submit a job to the node that sets the node to drain, does any local things needed, then exits. As part of the EpilogSlurmctld script, you could check for drained nodes

[slurm-users] slurmd and dynamic nodes

2022-09-23 Thread Groner, Rob
I'm working through how to use the new dynamic node features in order to take down a particular node, reconfigure it (using nvidia MIG to change the number of graphic cores available) and give it back to slurm. I'm at the point where I can take a node out of slurm's control from the master nod