Chiming in on Michael's suggestion. You can specify the same hostname in the slurm.conf but for the on-premise nodes you either set the DNS or the /etc/hosts entry to the local (=private) IP address. For the cloud nodes you set DNS or the hosts entry to the publicly reachable IP.
example /etc/hosts on-prem node: 10.10.10.10 myslurmcontroller example /etc/hosts cloud node: 50.50.50.50 myslurmcontroller example slurm.conf for both locations ControlMachine=myslurmcontroller ________________________________ From: slurm-users <slurm-users-boun...@lists.schedmd.com> on behalf of Sajesh Singh <ssi...@amnh.org> Sent: Friday, 22 January 2021 22:17 To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: [External] Re: [slurm-users] Cluster nodes on multiple cluster networks Thank you for the recommendation. Will try that out. Unfortunately the on-prem nodes cannot reach the head node via the public IP -Sajesh- From: slurm-users <slurm-users-boun...@lists.schedmd.com> On Behalf Of Michael Gutteridge Sent: Friday, January 22, 2021 3:18 PM To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: Re: [slurm-users] Cluster nodes on multiple cluster networks EXTERNAL SENDER I don't believe the IP address is required- if you can configure a DNS/hosts entry differently for cloud nodes you can set: SlurmCtldhost = controllername Then have "controllername" resolve to the private IP for the controller for the on-prem cluster, the public IP for the nodes in the cloud. Theoretically anyway- I haven't run a config like that and I'm not sure how the controller will react to such a configuration (i.e. getting slurm traffic on both interfaces). If the on-prem nodes can reach the public IP address of the controller it may be simpler to use only the public IP for the controller, but I don't know how your routing is set up. HTH - Michael On Fri, Jan 22, 2021 at 11:26 AM Sajesh Singh <ssi...@amnh.org<mailto:ssi...@amnh.org>> wrote: How would I deal with the address of the head node defined in the slurm.conf as I have it defined as SlurmctldHost=private-hostname(private.ip.addr) The private.ip.addr address is not reachable from the cloud nodes -Sajesh- From: slurm-users <slurm-users-boun...@lists.schedmd.com<mailto:slurm-users-boun...@lists.schedmd.com>> On Behalf Of Brian Andrus Sent: Friday, January 22, 2021 1:45 PM To: slurm-users@lists.schedmd.com<mailto:slurm-users@lists.schedmd.com> Subject: Re: [slurm-users] Cluster nodes on multiple cluster networks EXTERNAL SENDER You would need to have a direct connect/vpn so the cloud nodes can connect to your head node. Brian Andrus On 1/22/2021 10:37 AM, Sajesh Singh wrote: We are looking at rolling out cloud bursting to our on-prem Slurm cluster and I am wondering how to deal with the slurm.conf variable SlurmctldHost. It is currently configured with the private cluster network address that the on-prem nodes use to contact it. The nodes in the cloud would contact the head node via its public IP address. How can I configure Slurm so that both IPs are recognized as the head node? -Sajesh-
