Re: [slurm-users] Checking memory requirements in job_submit.lua

[Hendryk Bockelmann]

Hi,




/*
 * FIXME: Remove this in the future, lua can't handle 64bit
 * numbers!!!.  Use min_mem_per_node|cpu instead.
*/

Instead we check in job_submit.lua for s,th, like

 if (job_desc.min_mem_per_node ~= nil) and
    (job_desc.min_mem_per_node == 0) then

 end


But one can still check for

 if (job_desc.min_mem_per_cpu == 0) then

 end

Maybe this helps a bit.

CU,
Hendryk

On 14.06.2018 19:38, Prentice Bisbal wrote:

based on information given in job_submit_lua.c we decided not to use pn_min_memory any more. The comment in src says: slurm.log_user("minimum real mem per node specified as %u", job_desc.min_mem_per_node) For mem-per-cpu things are more confusing. Somehow min_mem_per_cpu = 2^63 = 0x8000000000000000 if sbatch/salloc does not set --mem-per-cpu, instead of being nil as expected ! slurm.log_user("minimum real mem per CPU specified as %u", job_desc.min_mem_per_cpu)

On 06/13/2018 01:59 PM, Prentice Bisbal wrote:

In my environment, we have several partitions that are 'general access', with each partition providing different hardware resources (IB, large mem, etc). Then there are other partitions that are for specific departments/projects. Most of this configuration is historical, and I can't just rearrange the partition layout, etc, which would allow Slurm to apply it's own logic to redirect jobs to the appropriate nodes.

To do this, I have added the following logic to my job_submit.lua script:

if job_desc.pn_min_mem > 65536 then

    job_desc.partition = 'mque'
    job_desc.qos = 'mque'
    return slurm.SUCCESS
end



mem per node = ntasks_per_node * ( ntasks_per_core / min_mem_per_cpu )

For the general access partitions, I've decided apply some of this logic in my job_submit.lua script. This logic would look at some of the job specifications and change the QOS/Partition for the job as appropriate. One thing I'm trying to do is have large memory jobs be assigned to my large memory partition, which is named mque for historical reasons.     slurm.user_msg("NOTICE: Partition switched to mque due to memory requirements.") This works when --mem is specified, doesn't seem to work when --mem-per-cpu is specified. What is the best way to check this when --mem-per-cpu is specified instead? Logically, one would have to calculate Is correct? If so, are there any flaws in the logic/variable names above? Also, is this quantity automatically calculated in Slurm by a variable that is accessible by job_submit.lua at this point, or do I need to calculate this myself?

Prentice

I've given up on calculating mem per node when --mem-per-cpu is specified. I was hoping to do this to protect my users from themselves, but the more I think about this, the more this looks like a fool's errand.

smime.p7s
Description: S/MIME Cryptographic Signature