[slurm-users] About sacct --format: how can I get info about the fields
Hello, guys, I am doing a parsing job on the output of the sacct command and I know that the —format option can specify the fields you'd like to be outputted. The difficulty I am facing is that I am in lack of info about the fields. For example, what are the possible values for those fields? What are the default values of the fields if slurm doesn’t have it? Is there any detailed documentation about those fields? Any help is greatly appreciated! Best Regards, Xiaojing
Re: [slurm-users] About sacct --format: how can I get info about the fields
Hello, slurm users and Brian, Thanks a lot for your reply. The thing is actually I know the fields. I just need to know detailed info about them. For example, you may get an “Unknown” for some time fields. And the maxVMSize field is an empty string except for some job steps. I need to know the possible values for each filed to do some calculation on them. But lack of info makes it very difficult. It seems like there is no documentation on it except for the web pages of sacct command. > Hello, guys, > > I am doing a parsing job on the output of the sacct command and I know that > the ?format option can specify the fields you'd like to be outputted. > The difficulty I am facing is that I am in lack of info about the fields. For > example, what are the possible values for those fields? What are the default > values of the fields if slurm doesn?t have it? Is there any detailed > documentation about those fields? > > Any help is greatly appreciated! > > Best Regards, > Xiaojing
Re: [slurm-users] About sacct --format: how can I get info about the fields
Hi, Marcus, Very grateful for you reply! I will try it out just as you suggested. Best Regards, Xiaojing > On Mar 5, 2021, at 3:59 PM, Marcus Boden wrote: > > Hi Xiaojing, > > my experience here is: you will have to try it out and see what works. At > least that's what I do whenever I parse sacct, as I did not find a detailed > description anywhere. The manpage is quite incomplete in that regard. > > Best, > Marcus > > On 05.03.21 03:02, xiaojingh...@163.com wrote: >> Hello, slurm users and Brian, >> Thanks a lot for your reply. The thing is actually I know the fields. I just >> need to know detailed info about them. For example, you may get an “Unknown” >> for some time fields. And the maxVMSize field is an empty string except for >> some job steps. I need to know the possible values for each filed to do some >> calculation on them. But lack of info makes it very difficult. >> It seems like there is no documentation on it except for the web pages of >> sacct command. >>> Hello, guys, >>> >>> I am doing a parsing job on the output of the sacct command and I know that >>> the ?format option can specify the fields you'd like to be outputted. >>> The difficulty I am facing is that I am in lack of info about the fields. >>> For example, what are the possible values for those fields? What are the >>> default values of the fields if slurm doesn?t have it? Is there any >>> detailed documentation about those fields? >>> >>> Any help is greatly appreciated! >>> >>> Best Regards, >>> Xiaojing > > -- > Marcus Vincent Boden, M.Sc. > Arbeitsgruppe eScience, HPC-Team > Tel.: +49 (0)551 201-2191, E-Mail: mbo...@gwdg.de > - > Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) > Am Faßberg 11, 37077 Göttingen, URL: https://www.gwdg.de > > Support: Tel.: +49 551 201-1523, URL: https://www.gwdg.de/support > Sekretariat: Tel.: +49 551 201-1510, Fax: -2150, E-Mail: g...@gwdg.de > > Geschäftsführer: Prof. Dr. Ramin Yahyapour > Aufsichtsratsvorsitzender: Prof. Dr. Norbert Lossau > Sitz der Gesellschaft: Göttingen > Registergericht: Göttingen, Handelsregister-Nr. B 598 > > Zertifiziert nach ISO 9001 > - >
[slurm-users] How can I get complete field values with without specify the length
Hi, guys I am doing a parsing job on slurm fields. Sometimes when one field is too long, slum will limit the length with a “+”. But I prefer to get the complete value of that field. Do you know how can I achieve that? I do not want to specify the length either coz it still cannot guarantee that you get the complete string. Any help will be greatly appreciated. Best regards, Xiaojing
Re: [slurm-users] How can I get complete field values with without specify the length
Hello, ole, Thank you for your reply. I forgot the -p option. I really works. Problem solved! Regards, Xiaojing > On Mar 9, 2021, at 2:56 PM, Ole Holm Nielsen > wrote: > > On 3/9/21 5:06 AM, xiaojingh...@163.com wrote: >> Hi, guys >> I am doing a parsing job on slurm fields. Sometimes when one field is too >> long, slum will limit the length with a “+”. But I prefer to get the >> complete value of that field. Do you know how can I achieve that? I do not >> want to specify the length either coz it still cannot guarantee that you get >> the complete string. > > You did not give examples of commands that you use. For the sacct command, > for example, you get complete output with -p or -P: > > -p, --parsable > output will be '|' delimited with a '|' at the end > -P, --parsable2 > output will be '|' delimited without a '|' at the end > > /Ole
Re: [slurm-users] How can I get complete field values with without specify the length
Hello, Mevik, I tried the -p option and it worked. Thank you for your reply! Regards, Xiaojing > On Mar 9, 2021, at 3:19 PM, Bjørn-Helge Mevik wrote: > > "xiaojingh...@163.com" writes: > >> I am doing a parsing job on slurm fields. Sometimes when one field is >> too long, slum will limit the length with a “+”. > > You don't say which slurm command you are trying to parse the output > from, but if it is sacctmgr, it has an option --parsable2(*) > specifically designed for parsing output, and which does not truncate > long field values. > > (*) There is also --parsable, but that puts an extra "|" at the end of > the line, so I prefer --parsable2. > > -- > Cheers, > Bjørn-Helge Mevik, dr. scient, > Department for Research Computing, University of Oslo >
[slurm-users] About seff command
Hi guys,
I would like to calculate the CPU efficiency and Memory efficiency of slurm
jobs. I know that the “seff” command is for this. I’d like to do the same job
on my own.
When I was studying the source code of the seff command, I saw the code below:
my $ncpus = 1;
if (exists $job->{'alloc_cpus'}) {
$ncpus = $job->{'alloc_cpus'};
} else {
if (exists $job->{'tres_alloc_str'}) {
$ncpus = Slurmdb::find_tres_count_in_string($job->{'tres_alloc_str'},
TRES_CPU);
}
}
I am wondering under what circumstance would the “alloc_cpus” field for a job
to be empty while having corresponding value in AllocCPU field?
Very grateful for any help.
Thank you!
[slurm-users] maxRSS and aveRSS
Hi guys, I would like to calculate the CPU efficiency and Memory efficiency of slurm jobs. I am having difficulty calculating the real “memory” a job use. According to slurm, “maxRSS” means "Maximum resident set size of all tasks in job”. If so, how can I get the memory used by a single job? As far as I am concerned, if I need to know the memory used by a single job/jobstep, I need to sum up the memory used for each task. So I think I should use the “aveRSS” field which gives the "average resident set size of all tasks in job”. If I multiply the “aveRSS” with “task”, I should get the real memory a job/jobstep used. But I studied the code of the “seff” command and it claims to be equivalent to "sacct -P -n -a --format JobID,User,Group,State,Cluster,AllocCPUS,REQMEM,TotalCPU,Elapsed,MaxRSS,ExitCode,NNodes,NTasks -j ”, which means I should use “maxRSS”. Can anyone give me some explanation on that? Very grateful for any help. Thank you! Regards, Xiaojing
