Re: [slurm-users] Reserve CPUs/MEM for GPUs

On Friday, 15 February 2019 7:35:45 AM PST Merlin Hartley wrote: > You could instead only allow the cpu partition to use 192G RAM and 20 CPU on > those nodes... This was basically how we did it at Swinburne, overlapping partitions where CPU jobs were restricted in a similar way and GPU jobs were

Re: [slurm-users] How to get the CPU usage of history jobs at each compute node?

On Friday, 15 February 2019 6:32:36 PM PST hu...@sugon.com wrote: > And I expect to get the cpu number of a certain history job on EACH compute > node. It's like a combination of the two above. I think you'd need to write your own monitoring tool then, I don't think Slurm captures that informati

Re: [slurm-users] How to get the CPU usage of history jobs at each compute node?

Thanks for Merlin Hartley and Eli V's replay ! This command(sacct -j ) can only get the total number of cpus for a history job: However , the command(scontrol show jobs --detail）can get the number of cpus of a running job on each node： And I expect to get the cpu number of a certain history jo

Re: [slurm-users] Reservation with memory

On 2/15/19 7:17 AM, Arnaud Renard URCA wrote: Does any of you have a solution to consider memory when creating a reservation ? I don't think memory is currently supported for reservations via TRES, it's certainly not listed in the manual page for scontrol either in 18.08 or in master (which

Re: [slurm-users] How to get the CPU usage of history jobs at each compute node?

sacct. Though, of course, accounting has to be turned on and working. On Fri, Feb 15, 2019 at 5:08 AM hu...@sugon.com wrote: > Dear there, > How to view the cpu usage of history jobs at each compute node? > However, this command(control show jobs jobid --detail) can only get the > cpu usage of t

Re: [slurm-users] Priority access for a group of users

Hi Paul, Marcus, Thank you for your replies. Using partition priority all makes sense. I was thinking of doing something similar with a set of nodes purchased by another group. That is, having a private high priority partition and a lower priority "scavenger" partition for the public. In this case

Re: [slurm-users] How to get the CPU usage of history jobs at each compute node?

using sacct [1] - assuming you have accounting [2] enabled: sacct -j Hope this helps! Merlin [1] https://slurm.schedmd.com/sacct.html [2] https://slurm.schedmd.com/accounting.html -- Merlin Hartley Computer

Re: [slurm-users] How to request ONLY one CPU instead of one socket or one node?

Seems like you aren't specifying a --mem option, so the default would be to ask for a whole-node’s worth of RAM thus you would use the whole node for each job. Hope this is useful! Merlin -- Merlin Hartley Computer Officer MRC Mitochondrial Biology Unit University of Cambridge Cambridge, CB2 0X

Re: [slurm-users] Reserve CPUs/MEM for GPUs

You could instead only allow the cpu partition to use 192G RAM and 20 CPU on those nodes... -- Merlin Hartley > On 13 Feb 2019, at 07:38, Quirin Lohr wrote: > > Hi all, > > we have a slurm cluster running on nodes with 2x18 cores, 256GB RAM and 8 > GPUs. Is there a way to reserve a bare min

[slurm-users] Reservation with memory

Dear All, Does any of you have a solution to consider memory when creating a reservation ? Thanks Arnaud RENARD Arnaud RENARD Université de Reims Champagne-Ardenne Chef de projet SiMSEO Nord&Est et Centre de Calcul Régional ROMEO Ingénieur de Recherche en Informatiq

Re: [slurm-users] Priority access for a group of users

Yup, PriorityTier is what we use to do exactly that here.  That said unless you turn on preemption jobs may still pend if there is no space.  We run with REQUEUE on which has worked well. -Paul Edmon- On 2/15/19 7:19 AM, Marcus Wagner wrote: Hi David, as far as I know, you can use the Prio

Re: [slurm-users] How to request ONLY one CPU instead of one socket or one node?

Hello, Are you sure you're NOT getting 1 CPU when you run your job? You might want to put some echo logic into your job to look at Slurm env variables of the node your job lands on as a way of checking. E.g.: echo $SLURM_CPUS_ON_NODE echo $SLURM_JOB_CPUS_PER_NODE I don't see anything wrong with

Re: [slurm-users] Priority access for a group of users

Hi David, as far as I know, you can use the PriorityTier (partition parameter) to achieve this. According to the manpages (if I remember right) jobs from higher priority tier partitions have precedence over jobs from lower priority tier partitions, without taking the normal fairshare priority

[slurm-users] How to get the CPU usage of history jobs at each compute node?

Dear there, How to view the cpu usage of history jobs at each compute node? However, this command(control show jobs jobid --detail) can only get the cpu usage of the currently running job at each compute node : Appreciatively, Menglong

[slurm-users] Priority access for a group of users

Hello. We have a small set of compute nodes owned by a group. The group has agreed that the rest of the HPC community can use these nodes providing that they (the owners) can always have priority access to the nodes. The four nodes are well provisioned (1 TByte memory each plus 2 GRID K2 graph