Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Yakupov, Renat /DZNE
Thank you for the suggestion, Bill. I will take a look at the Launcher. Best, Renat. From: slurm-users [slurm-users-boun...@lists.schedmd.com] On Behalf Of Bill Barth [bba...@tacc.utexas.edu] Sent: Friday, November 10, 2017 2:05 PM To: Slurm User Communit

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Bill Barth
Renat, Not to toot our own horn too much, but TACC develops a tool designed for launching lots of individual serial tasks on a parallel system and distributing them well inside a single scheduler job. You might check out: https://github.com/TACC/launcher. There’s no support for the tool, but it

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Yakupov, Renat /DZNE
Hi Gennaro, they are all tasks within the same job... If I start multiple jobs, I can just use --begin. I think I found a solution using shell PID $$, which is unique for each task. Best, Renat. From: slurm-users [slurm-users-boun...@lists.schedmd.com] O

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Gennaro Oliva
Hi Renat, On Fri, Nov 10, 2017 at 10:03:37AM +0100, Yakupov, Renat /DZNE wrote: > slurm 2.5.0! seeing today's announcement about a double digit version > release, that is... ancient! according to the NEWS file, native jobarray support was introduced in version 2.6.0pre1 ... bad luck. You can sti

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Yakupov, Renat /DZNE
Hi John, I am processing high-resolution MRI data. The processing pipeline itself is in Matlab. I will suggest updating Slurm, but seeing how far behind we are, it seems to me Slurm is really not a priority for them... Best, Renat. From: slurm-users [slu

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Gennaro Oliva
Hi Renat, On Fri, Nov 10, 2017 at 09:44:06AM +0100, Yakupov, Renat /DZNE wrote: > How can I find that out? Dont see anything like that in slurm.conf... srun -V Regards, -- Gennaro Oliva

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread John Hearns
Renat, I understand better here. It does look like job arrays are what you need. Maybe you could ask for the Slurm version on your HPC to be updated. I gues that may be difficult. I am also guessing that you are doing some sort of Big Data task, maybe in Python? On 10 November 2017 a

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Yakupov, Renat /DZNE
slurm 2.5.0! seeing today's announcement about a double digit version release, that is... ancient! Best, Renat. From: slurm-users [slurm-users-boun...@lists.schedmd.com] On Behalf Of Loris Bennett [loris.benn...@fu-berlin.de] Sent: Friday, November 10, 20

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Yakupov, Renat /DZNE
Thank you for suggestions, John. I am a user though, I cant install anything. The first problem is not even access to the data, but to a "job list" file. When I started working on these data a month or so ago, I got access a system with several compute nodes (and eventually also an HPC). So I tho

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Loris Bennett
"Yakupov, Renat /DZNE" writes: > How can I find that out? Dont see anything like that in slurm.conf... squeue --version > Best, > Renat. > > From: slurm-users [slurm-users-boun...@lists.schedmd.com] On Behalf Of > Gennaro Oliva [oliv...@na.icar.cnr.it]

Re: [slurm-users] introduce short delay starting multiple parallel jobs with srun

2017-11-10 Thread Yakupov, Renat /DZNE
How can I find that out? Dont see anything like that in slurm.conf... Best, Renat. From: slurm-users [slurm-users-boun...@lists.schedmd.com] On Behalf Of Gennaro Oliva [oliv...@na.icar.cnr.it] Sent: Thursday, November 09, 2017 3:55 PM To: Slurm User Commun