Re: [slurm-users] Header lengths are longer than data received after changing SelectType & GresTypes to use MPS

2020-04-07 Thread Christopher Samuel
On 4/7/20 2:48 PM, Robert Kudyba wrote: How can I get this to work by loading the correct Bright module? You can't - you will need to recompile Slurm. The error says: Apr 07 16:52:33 node001 slurmd[299181]: fatal: We were configured to autodetect nvml functionality, but we weren't able to f

Re: [slurm-users] Header lengths are longer than data received after changing SelectType & GresTypes to use MPS

2020-04-07 Thread Robert Kudyba
> Apr 07 16:52:33 node001 slurmd[299181]: fatal: We were configured to > autodetect nvml functionality, but we weren't able to find that lib when > Slurm was configured. > > > > Apparently the Slurm build you are using has not be compiled against NVML > and as such it cannot use the autodetect func

Re: [slurm-users] Header lengths are longer than data received after changing SelectType & GresTypes to use MPS

2020-04-07 Thread Davide Vanzo
I do not have experience with using NVML and MPS, but here are my thoughts. What I would focus my attention is this line: Apr 07 16:52:33 node001 slurmd[299181]: fatal: We were configured to autodetect nvml functionality, but we weren't able to find that lib when Slurm was configured. Apparentl

Re: [slurm-users] Header lengths are longer than data received after changing SelectType & GresTypes to use MPS

2020-04-07 Thread Robert Kudyba
OK when restarting slurmd on the nodes I get these errors: Apr 07 16:52:33 node001 systemd[1]: Starting Slurm node daemon... Apr 07 16:52:33 node001 slurmd[299181]: Message aggregation disabled Apr 07 16:52:33 node001 slurmd[299181]: WARNING: A line in gres.conf for GRES mps has 400 more configure

Re: [slurm-users] Header lengths are longer than data received after changing SelectType & GresTypes to use MPS

2020-04-07 Thread Davide Vanzo
Robert, That error is typically due to slurmd/slurmctld version mismatch or different configuration. I would not be surprised if you need to restart slurmd too after changing the SelectType configuration. Also, do not forget this warning from the documentation when it comes to modifying SelectT

[slurm-users] Header lengths are longer than data received after changing SelectType & GresTypes to use MPS

2020-04-07 Thread Robert Kudyba
Using Slurm 20.02 on CentIOS 7.7 with Bright Cluster. We changed the following options to enable MPS: SelectType=select/cons_tres GresTypes=gpu,mic,mps I restarted slurmctld and ran scontrol reconfigure, however all jobs get the below error: [2020-04-07T15:29:00.741] debug: backfill: no jobs to b

Re: [slurm-users] Error buildind rpm on Centos 7

2020-04-07 Thread Ole Holm Nielsen
Hi Alfonso, You just need to get the CentOS 7 prerequisites right, check out my Slurm installation Wiki page: https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#build-slurm-rpms HTH, Ole On 07-04-2020 13:07, Alfonso Núñez Slagado wrote:     I'm trying to build rpm packages running follow

Re: [slurm-users] Error buildind rpm on Centos 7

2020-04-07 Thread William Brown
Search the list archive, I had the same and it was because I had MariaDB installed but as the packaging of MariaDB changed I was missing a required RPM. They split it differently and there is another RPM prerequisite. Can't recall the name just now, but search the archive. William On Tue, 7 Apr

[slurm-users] Error buildind rpm on Centos 7

2020-04-07 Thread Alfonso Núñez Slagado
Hi all,     I'm trying to build rpm packages running following comands but i get allways the same error. I've checked the BUILD config.log and /usr/bin/mysql_config seems to be found... Any clue? rpmbuild -ta slurm-20.02.1.tar.bz2 or rpmbuild -ta slurm-19.05.6.tar.bz2   RPM build errors: