Snakemake is a tool to manage job dependencies. I don't have experience
using this in a slurm context, but I have used Snakemake to manage
complex depencies in workflows in other contexts. Snakemake should
support slurm.
HTH,
Jan
On 02-03-2021 20:16, Jason Simms wrote:
Hello all,
I am relatively new to the nuances of handling complex dependencies in
Slurm, so I'm hoping the hive mind can help. I have a user wanting to
accomplish the following:
* submit one job
* submit multiple jobs that are dependent on the output from the first
job (so they just need to launch once the first job has completed)
* submit one final job dependent on all the previous jobs completing
Is there a way to do this cleanly? So it's a three stage process. I have
ideas in my head of writing Slurm JobIDs to a file, reading them out,
and managing dependencies that way, but perhaps there is a more
efficient way (or perhaps not!).
Warmest regards,
Jason
--
*Jason L. Simms, Ph.D., M.P.H.*
Manager of Research and High-Performance Computing
XSEDE Campus Champion
Lafayette College
Information Technology Services
710 Sullivan Rd | Easton, PA 18042
Office: 112 Skillman Library
p: (610) 330-5632
