Hi slurm-users, Hoping you can point me in the right direction….I am trying to setup the Slurm REST API under parallelcluster v3.1.1. I am following the instructions at https://slurm.schedmd.com/jwt.html. I have added the AuthAltTypes=rest_auth/jwt and the AuthAultParameters=jwt_key=… to /opt/slum/etc/slurm.conf but restarting slurmctld fails with:
slurmctld: debug: slurmctld log levels: stderr=debug3 logfile=debug3 syslog=quiet slurmctld: debug: Log file re-opened slurmctld: pidfile not locked, assuming no running daemon slurmctld: Stack size set to 10485760 slurmctld: debug: slurmscriptd: Got ack from slurmctld, initialization successful slurmctld: debug: slurmctld: slurmscriptd fork()'d and initialized. slurmctld: slurmctld version 21.08.5 started on cluster parallelcluster slurmctld: debug3: Trying to load plugin /opt/slurm/lib/slurm/cred_munge.so slurmctld: debug: _slurmscriptd_mainloop: started slurmctld: debug3: Called _msg_readable slurmctld: debug: _slurmctld_listener_thread: started listening to slurmscriptd slurmctld: cred/munge: init: Munge credential signature plugin loaded slurmctld: debug3: Success. slurmctld: debug3: Called _msg_readable slurmctld: debug3: Trying to load plugin /opt/slurm/lib/slurm/auth_munge.so slurmctld: debug: auth/munge: init: Munge authentication plugin loaded slurmctld: debug3: Success. slurmctld: debug3: Trying to load plugin /opt/slurm/lib/slurm/rest_auth_jwt.so slurmctld: debug3: Couldn't find sym 'auth_p_create' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_destroy' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_verify' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_get_uid' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_get_gid' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_get_host' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_pack' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_unpack' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_thread_config' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_thread_clear' in the plugin slurmctld: debug3: Couldn't find sym 'auth_p_token_generate' in the plugin slurmctld: fatal: plugin_load_and_link: Plugin loading failed due to missing symbols. Plugin is corrupted. Looks like I am missing something when trying to enable JWT authentication with the slurm services. I am using the slurm install that comes out of the box with parallelcluster 3.1.1. I was wondering if there are any suggestions about how to fix/resolve this error?. Thanks. Slurmrestd auth types: $ slurmrestd -a list slurmrestd: Possible REST authentication plugins: slurmrestd: rest_auth/local slurmrestd: rest_auth/jwt Also, step 2 in the JWT instructions at https://slurm.schedmd.com/jwt.html is a little confusing – the key is saved to the StateSaveLocation but the rest of the instructions reference /etc/slurm? Am I missing something here (likely!) ? dd if=/dev/random of=/var/spool/slurm/statesave/jwt_hs256.key bs=32 count=1 chown slurm:slurm /etc/slurm/jwt_hs256.key chmod 0600 /etc/slurm/jwt_hs256.key chown root:root /etc/slurm chmod 0755 /etc/slurm Thanks! Regards, j
