Hi Alper,
On 18-08-2023 18:39, Alper Alimoglu wrote:
In slurm we can build pipelines using [slurm dependencies][1], which
allows us to run workflows.
In my work, I have stuck in a point regarding finding a workflow that I
can run using Slurm.
As an example, I have to use a workflow benchmark like in here
https://pegasus.isi.edu/workflow_gallery/
<https://pegasus.isi.edu/workflow_gallery/> , but all of them are
implemented inside Pegasus.
I was wondering is there any public scientific-workflow examples that
can be run through Slurm?
Any suggestion is highly appreciated.
[1]:
https://www.hpc.caltech.edu/documentation/faq/dependencies-and-pipelines
My colleagues at Technical University of Denmark are heavy users of
workflows through Slurm on our cluster, and this accounts for most of
our usage. They have developed an Open Source workflow system:
MyQueue is a frontend for SLURM/PBS/LSF that makes handling of tasks easy. It
has a command-line interface called mq with a number of Sub-commands and a
Python interface for managing Workflows. Simple to set up: no system
administrator or database required.
See https://myqueue.readthedocs.io/en/latest/
I'm personally not involved in MyQueue, but you might take a look to see
if it's useful in your environment.
Best regards,
Ole
