I think this is all true, but...
There's a strong theme of "choose your own adventure" to designing performant
OpenStack infrastructure. Consider it as a continuum of convenience vs
performance: at one end we get all the flexibility of cloud and at the other we
get all the performance of bare
I think that’s certainly true. As with all things, it depends on your
workloads. The vast majority of our genomics codes are either single threaded,
or multi-threaded on a single node. There's relatively little MPI, and we
maintain a "parallel" queue on bare metal for precisely that set of wo
Hi,
i suggest you not only look at the flexibility / complexity regarding
administrating a cluster with OpenStack [there are also many other tools
for that] but also *actually benchmark* with a parallel (threaded) code
you know well and check the strong scaling via Amdhal's law by making some
spee
Hi Chris,
thanks for your sentiments. Like most things, you have two sides of a coin:
flexibility, which is what we all want, and complexity, which is the price you
need to pay for the flexibility.
This is why I thought it is best to ask the community for first hand
experiences. One thing we
Hi Tim,
many thanks for sharing your experiences here and sorry for my slow reply, I
am currently on annual leave and thus don't check emails on a daily base.
The additional layer of complexity is probably the price you have to pay for
the flexibility. What we are aiming for is having a more f
Hi John,
thanks for the links. I know Martyn personally and of course I know his
(in)famous talks at the CIUK workshops.
The InfiniBand is not for storage (I still consider using IB for storage a
waste) it is for parallel computing where you need the low latency. As far as
I can see even the l
Here, we deploy some clusters on OpenStack, and some traditionally as bare
metal. Our largest cluster is actually a mixture of both, so we can
dynamically expand it from the OpenStack service when needed.
Our aim eventually is to use OpenStack as a common deployment layer, even for
the bare m
At Job$1 we run multiple clusters on top on openstack. We are a very
interactive HPC shop and it really helps to deliver things that we couldn't
easily do any other way. The cgroups side of things is used pretty heavily
but it doesn't always address contention in the way a dedicated VM can. Our
net
On 29/6/20 5:09 pm, Jörg Saßmannshausen wrote:
we are currently planning a new cluster and this time around the idea was to
use OpenStack for the HPC part of the cluster as well.
I was wondering if somebody has some first hand experiences on the list here.
At $JOB-2 I helped a group set up a
Jorg, I would back up what Matt Wallis says. What benefits would Openstack
bring you ?
Do you need to set up a flexible infrastructure where clusters can be
created on demand for specific projects?
Regarding Infiniband the concept is SR-IOV. This article is worth reading:
https://docs.openstack.or
Hi Jörg,
you may want to talk to Kenneth Tan @ https://www.sardinasystems.com/
In any case, he should be able to clarify your question regarding IB.
Happy to provide his email address.
Best
Igor
On Tue, 30 Jun 2020 at 12:22, Jörg Saßmannshausen <
sassy-w...@sassy.formativ.net> wrote:
> Dear all,
The video is here. From 04:00 onwards
https://fosdem.org/2020/schedule/event/magic_castle/
"OK your cluster will be available in about 20 minutes"
On Tue, 30 Jun 2020 at 14:27, INKozin wrote:
> And that's how you deploy an HPC cluster!
>
> On Tue, 30 Jun 2020 at 14:21, John Hearns wrote:
>
>>
I saw Magic Castle being demonstrated lve at FOSDEM this year.
It is more a Terraform/ansible setup for configuring clusters on demand.
The person demonstrating it called a Google Home assistant with a voice
command and asked it to build and deploy a cluster - which it did!
On Tue, 30 Jun 2020 at
Hi Jörg
> On 30 Jun 2020, at 10:09 am, Jörg Saßmannshausen
> wrote:
>
> Dear all,
>
> we are currently planning a new cluster and this time around the idea was to
> use OpenStack for the HPC part of the cluster as well.
>
> I was wondering if somebody has some first hand experiences on the
Dear all,
we are currently planning a new cluster and this time around the idea was to
use OpenStack for the HPC part of the cluster as well.
I was wondering if somebody has some first hand experiences on the list here.
One of the things we currently are not so sure about it is InfiniBand (or
15 matches
Mail list logo
