Hi Paddy, Thanks for the comments and suggestions!
__________________________________________________ *Jacob D. Chappell, CSM* *Research Computing Associate* Research Computing | Research Computing Infrastructure Information Technology Services | University of Kentucky jacob.chapp...@uky.edu On Tue, May 28, 2019 at 12:56 PM Paddy Doyle <pa...@tchpc.tcd.ie> wrote: > Hi Jacob, > > On Tue, May 28, 2019 at 11:38:23AM -0400, Jacob Chappell wrote: > > > Hello all, > > > > Is it possible in Slurm to check RawUsage against GrpTRESMins and > prevent a > > job from being submitted if the RawUsage exceeds the GrpTRESMins? My > center > > needs this feature for detailed accounting constraints. The RawUsage is > > important to us because we weight certain resource types and want to bill > > people appropriately (e.g., if you use 12 GB of RAM and 1 CPU you should > be > > billed for 2 CPUs). > > That sounds like you want 'AccountingStorageEnforce=safe'; see: > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fslurm.schedmd.com%2Faccounting.html&data=02%7C01%7Cjacob.chappell%40uky.edu%7Cc75abaa6578f450426cf08d6e38d6434%7C2b30530b69b64457b818481cb53d42ae%7C0%7C0%7C636946593746913338&sdata=NvSlTWVW1FLzuU70n2h%2BuUkynFO3cnITbx%2BXRdTIBaI%3D&reserved=0 > > And you probably want to think about PriorityDecayHalfLife and > PriorityUsageResetPeriod, to control what happens after someone does hit > the limit. > > That won't prevent the jobs from being submitted though (if I read your > question correctly); it will stop them from running. > > We have written a little add-on 'slurm-bank' which ties together the > RawUsage from sshare (in seconds) and GrpTRESMins from sacctmgr (in > minutes) into a single balance statement: > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjcftang%2Fslurm-bank&data=02%7C01%7Cjacob.chappell%40uky.edu%7Cc75abaa6578f450426cf08d6e38d6434%7C2b30530b69b64457b818481cb53d42ae%7C0%7C0%7C636946593746913338&sdata=5T88ku58LYYJ4BK7jOzpnUVl%2Bvq3SmCjT2bLS2M0gjQ%3D&reserved=0 > > Note that it only does simple cpu usage counting and doesn't support > weights for different amounts of RAM+CPU combinations. So it may not be > useful for you. > > Thanks, > Paddy > > > I am thinking about implementing a custom Slurm accounting plugin, but > > wanted to check to see if this feature existed already first. Or, perhaps > > the feature is a work in progress and planned for a future release? Are > > there any upcoming Slurm changes that might break such a plugin if a > custom > > implementation is necessary? > > > > Thanks, > > __________________________________________________ > > *Jacob D. Chappell, CSM* > > *Research Computing Associate* > > Research Computing | Research Computing Infrastructure > > Information Technology Services | University of Kentucky > > jacob.chapp...@uky.edu > > -- > Paddy Doyle > Trinity Centre for High Performance Computing, > Lloyd Building, Trinity College Dublin, Dublin 2, Ireland. > Phone: +353-1-896-3725 > > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.tchpc.tcd.ie%2F&data=02%7C01%7Cjacob.chappell%40uky.edu%7Cc75abaa6578f450426cf08d6e38d6434%7C2b30530b69b64457b818481cb53d42ae%7C0%7C0%7C636946593746913338&sdata=k%2Bc9k70noPzfWANABUu8jgBDJ1Gue6qV72aFD4p8xAA%3D&reserved=0 > >
