Re: [slurm-users] Array jobs vs. many jobs

2019-11-24 Thread Mark Hahn
Quick question that I'm not sure how to find the answer to otherwise: do array jobs have less impact on the scheduler in any way than a whole long list of jobs run the more traditional way? Less startup overhead, anything like that? kinda sorta. I think of array jobs as a bit like Python generat

Re: [slurm-users] Array jobs vs. many jobs

2019-11-22 Thread Christopher Samuel
Hi Ryan, On 11/22/19 12:18 PM, Ryan Novosielski wrote: Quick question that I'm not sure how to find the answer to otherwise: do array jobs have less impact on the scheduler in any way than a whole long list of jobs run the more traditional way? Less startup overhead, anything like that? Slu

Re: [slurm-users] Array jobs vs. many jobs

2019-11-22 Thread Ree, Jan-Albert van
Jan-Albert van Ree | Linux System Administrator | Digital Services MARIN | T +31 317 49 35 48 | mailto:j.a.v@marin.nl | http://www.marin.nl It helps a lot indeed ; we run arrays up to 100k elements and more. If you submit 100k separate jobs, the scheduler will definately grind to a halt.