Interesting thing I found! As I checked the log, I saw part_policy_valid_acct: job's account not permitted to use this partition (RUBY allows Y8 not y8)
However, in the command I use "-A Y8" and I am sure about that. The parts file contains PartitionName=RUBY AllowAccounts=Y8 Nodes=compute-0-[2-4] So, I decided to define y8 instead of Y8. The parts file then looks PartitionName=RUBY AllowAccounts=y8 Nodes=compute-0-[2-4] and when I run "-A y8", I don't get that error. Seems to be a bug. If there is a reason for that, please let me know. Regards, Mahmood On Wed, May 16, 2018 at 3:33 PM, Mahmood Naderan <mahmood...@gmail.com> wrote: > Yes they are the same. > > [root@rocks7 ~]# cp /etc/slurm/slurm.conf rocks7 > [root@rocks7 ~]# scp compute-0-0:/etc/slurm/slurm.conf compute-0-0 > slurm.conf > 100% 2465 3.6MB/s 00:00 > [root@rocks7 ~]# scp compute-0-1:/etc/slurm/slurm.conf compute-0-1 > slurm.conf > 100% 2465 4.7MB/s 00:00 > [root@rocks7 ~]# md5sum rocks7 compute-0-* > 41df7afb1ed37cc24d8151dc8d7e6c1e rocks7 > 41df7afb1ed37cc24d8151dc8d7e6c1e compute-0-0 > 41df7afb1ed37cc24d8151dc8d7e6c1e compute-0-1 > > > > The cpu limit on ruby partition is 20 cores. The nodes in that > partition are Intel Xeons with the following specs > > [root@rocks7 ~]# rocks run host compute-0-5 "lscpu" > Warning: untrusted X11 forwarding setup failed: xauth key data not generated > Architecture: x86_64 > CPU op-mode(s): 32-bit, 64-bit > Byte Order: Little Endian > CPU(s): 56 > On-line CPU(s) list: 0-55 > Thread(s) per core: 2 > Core(s) per socket: 14 > Socket(s): 2 > NUMA node(s): 2 > ... > > There are 14 physical cores on each cpu and therefore 28 physical > cores which means 56 threads. The requested cores are less that 28 so, > it should be ok. I don't know why slurm said that error. > > > Regards, > Mahmood > > > > > On Wed, May 16, 2018 at 1:48 PM, John Hearns <hear...@googlemail.com> wrote: >> Mahmood, >> you should check that the slurm.conf files are identical on the head node >> and the compute nodes after you run the rocks sync. >> >>
