A lot of production HPC runs on cloud systems.
AWS is big for this via their AWS Parallelcluster stack which does
include lustre support via vfXT for lustre service although they are
careful to caveat it as staging/scratch space not suitable for
persistant storage. AWS has some cool node types now with 25gig, 50gig
and 100-gigabit network support.
Microsoft Azure is doing amazing things now that they have the
cyclecomputing folks on board, integrated and able to call shots within
the product space. They actually offer bare metal HPC and infiniband
SKUs now and have some interesting parallel filesystem offerings as well.
Can't comment on google as I've not touched or used it professionally
but AWS and Azure for sure are real players now to consider if you have
an HPC requirement.
That said, however, a sober cost accounting still shows on-prem or
"owned' HPC is best from a financial perspective if your workload is
24x7x365 constant. The cloud based HPC is best for capability, bursty
workloads, temporary workloads, auto-scaling, computing against
cloud-resident data sets or the neat new model where instead of on-prem
multi-user shared HPC you go out and decide to deliver individual
bespoke HPC clusters to each user or team on the cloud.
The big paradigm shift for cloud HPC is that it does not make a lot of
sense to make a monolithic stack shared by multiple competing users and
groups. The automated provisioning and elasticity of the cloud make it
more sensible to build many clusters so that you can tune each cluster
specifically for the cluster or workload and then blow it up when the
work is done.
My $.02 of course!
Chris
Jonathan Aquilina <mailto:jaquil...@eagleeyet.net>
July 22, 2019 at 1:48 PM
Hi Guys,
I am looking at
https://cloud.google.com/blog/products/storage-data-transfer/introducing-lustre-file-system-cloud-deployment-manager-scripts
This basically allows you to deploy a lustre cluster on google cloud.
In your HPC setups have you considered moving towards cloud based
clusters?
Regards,
Jonathan
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
