Hi folks,
Those who know me from my day job and other email address know that I've
been a hardcore SGE person for a decade+ now. Feels bad to even write
this email, heh
Internally at my company we've been having a great internal discussion
about SGE and SLURM, the specific trigger being our widespread use of
AWS Parallelcluster for auto-scaling compute farms on AWS and the
decision by AWS to deprecate the open source SGE distributed with the
stack at some point in the future.
Related to this is the longstanding use of SGE in the life sciences --
there are genome sequencers and other wet lab instruments that ship
natively with SGE support from the vendor so there is going to be a long
tail of SGE still in use or targeted for use in biotech and pharma spaces.
I think there is still a future for commercial SGE especially after the
Univa -> Altair tie up but that still leaves the poor orphaned/forked
open source SGE distro's still kinda hanging out there with no real
updates or improvements in ages so I am understanding of the AWS HPC
folk desire to pare down their supported scheduler stack.
I want to build up my own knowledge and prep for our own increased use
of SLURM on AWS. I'm very comfortable with SGE architecture,
operational philosophy and capabilities but I lack similar info for
modern SLURM. I'm ready and willing to start from scratch to build my
own transition and "differences between SGE/SLURM" documentation but was
wondering who out there has made this transition before and if there are
any public domain FAQs, wikis, technical writeups or other guidance that
I can learn from.
If I can manage to put my own materials together and they look sensible
I will plan on publishing them openly. Thanks!
So far the internal conversation we are having is centering on the
differences in resource based job scheduling when there are specific
needs to declare required resources up front like GPUs or memory
requirements. Most of the differences beyond basic queue/partition
design seem to center around the minutiae of scheduling and placing jobs
but I'm sure I'm missing other larger areas.
Regards
Chris
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
