Double-check you have all the packages.
When slurm is built, slurmrestd is a separate package and is only built
if the whole set was directed to do so. If they did not build it, you
will need to do so yourself. This will mean using your custom built
files throughout.
Brian Andrus
On 3/7/2022 3:04 PM, Justin Freeman wrote:
Hi slurm-users,
Hoping you can point me in the right direction….I am trying to setup
the Slurm REST API under parallelcluster v3.1.1. I am following the
instructions at https://slurm.schedmd.com/jwt.html. I have added the
AuthAltTypes=rest_auth/jwt and the AuthAultParameters=jwt_key=… to
/opt/slum/etc/slurm.conf but restarting slurmctld fails with:
slurmctld: debug: slurmctld log levels: stderr=debug3 logfile=debug3
syslog=quiet
slurmctld: debug: Log file re-opened
slurmctld: pidfile not locked, assuming no running daemon
slurmctld: Stack size set to 10485760
slurmctld: debug: slurmscriptd: Got ack from slurmctld,
initialization successful
slurmctld: debug: slurmctld: slurmscriptd fork()'d and initialized.
slurmctld: slurmctld version 21.08.5 started on cluster parallelcluster
slurmctld: debug3: Trying to load plugin
/opt/slurm/lib/slurm/cred_munge.so
slurmctld: debug: _slurmscriptd_mainloop: started
slurmctld: debug3: Called _msg_readable
slurmctld: debug: _slurmctld_listener_thread: started listening to
slurmscriptd
slurmctld: cred/munge: init: Munge credential signature plugin loaded
slurmctld: debug3: Success.
slurmctld: debug3: Called _msg_readable
slurmctld: debug3: Trying to load plugin
/opt/slurm/lib/slurm/auth_munge.so
slurmctld: debug: auth/munge: init: Munge authentication plugin loaded
slurmctld: debug3: Success.
slurmctld: debug3: Trying to load plugin
/opt/slurm/lib/slurm/rest_auth_jwt.so
slurmctld: debug3: Couldn't find sym 'auth_p_create' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_destroy' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_verify' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_get_uid' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_get_gid' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_get_host' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_pack' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_unpack' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_thread_config' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_thread_clear' in the plugin
slurmctld: debug3: Couldn't find sym 'auth_p_token_generate' in the plugin
slurmctld: fatal: plugin_load_and_link: Plugin loading failed due to
missing symbols. Plugin is corrupted.
Looks like I am missing something when trying to enable JWT
authentication with the slurm services. I am using the slurm install
that comes out of the box with parallelcluster 3.1.1. I was wondering
if there are any suggestions about how to fix/resolve this error?.
Thanks.
Slurmrestd auth types:
$ slurmrestd -a list
slurmrestd: Possible REST authentication plugins:
slurmrestd: rest_auth/local
slurmrestd: rest_auth/jwt
Also, step 2 in the JWT instructions at
https://slurm.schedmd.com/jwt.html is a little confusing – the key is
saved to the StateSaveLocation but the rest of the instructions
reference /etc/slurm? Am I missing something here (likely!) ?
dd if=/dev/random of=/var/spool/slurm/statesave/jwt_hs256.key bs=32
count=1
chown slurm:slurm /etc/slurm/jwt_hs256.key
chmod 0600 /etc/slurm/jwt_hs256.key
chown root:root /etc/slurm
chmod 0755 /etc/slurm
Thanks!
Regards,
j
