Re: [Beowulf] flatpack

2019-07-22 Thread John Hearns via Beowulf
Having used Snaps on Ubuntu - which seems to be their preferred method of distributing some applications, I have a slightly different take on the containerisation angle and would de-emphaise that. My take is that snaps/flatpak attack the "my distro ships with gcc version 4.1 but I need gcc version

Re: [Beowulf] flatpack

2019-07-22 Thread Chris Samuel
On 22/7/19 10:40 pm, Jonathan Aquilina wrote: So in a nut shell this is taking dockerization/ containerization and making it more for the every day Linux user instead of the HPC user? I don't think this goes as far as containers with isolation, as I think that's not what they're trying to do.

Re: [Beowulf] Lustre on google cloud

2019-07-22 Thread Jonathan Aquilina
Guess at some point I will need to fire up a test cluster and try things out. But usually from my experiences with the three major cloud providers you have access obviously with limited means to make modifications so I am curious to know how easy it would be to tweak settings. I have a bit of ex

Re: [Beowulf] Lustre on google cloud

2019-07-22 Thread Chris Samuel
On 22/7/19 10:31 pm, Jonathan Aquilina wrote: I am sure though that with the GUI side of things through the console I am sure it makes things a lot easier to setup and manage no? You would hope so! Although I've got to say with my limited experience of Lustre when you're running it you pret

Re: [Beowulf] flatpack

2019-07-22 Thread Jonathan Aquilina
So in a nut shell this is taking dockerization/ containerization and making it more for the every day Linux user instead of the HPC user? It would be interesting to have a distro built around such a setup. Regards, Jonathan -Original Message- From: Beowulf On Behalf Of Chris Samuel Sen

Re: [Beowulf] flatpack

2019-07-22 Thread Chris Samuel
On 22/7/19 10:26 pm, Jonathan Aquilina wrote: Hi Guys, I think I might be a bit tardy to the party here, but the way you describe flatpack is equivalent to the portable apps on windows is my understanding correct? It seems that way, with an element of sandboxing to try and protect the user w

Re: [Beowulf] Lustre on google cloud

2019-07-22 Thread Jonathan Aquilina
I am based in Europe so that answers my question partially. I am sure though that with the GUI side of things through the console I am sure it makes things a lot easier to setup and manage no? Regards, Jonathan -Original Message- From: Beowulf On Behalf Of Chris Samuel Sent: Tuesday, 2

Re: [Beowulf] Lustre on google cloud

2019-07-22 Thread Chris Samuel
On 22/7/19 10:12 pm, Jonathan Aquilina wrote: I am aware of that as I follow their youtube channel. Fair enough, others may not. :-) I think my main query is compared to managing a cluster in house is this the way forward be it AWS or google cloud? I think the answer there is likely "it d

Re: [Beowulf] flatpack

2019-07-22 Thread Jonathan Aquilina
Hi Guys, I think I might be a bit tardy to the party here, but the way you describe flatpack is equivalent to the portable apps on windows is my understanding correct? Regards, Jonathan From: Beowulf On Behalf Of Jonathan Engwall Sent: Monday, 22 July 2019 19:59 To: Christopher Samuel Cc: Beo

Re: [Beowulf] Lustre on google cloud

2019-07-22 Thread Jonathan Aquilina
Hi Chris, I am aware of that as I follow their youtube channel. I think my main query is compared to managing a cluster in house is this the way forward be it AWS or google cloud? Regards, Jonathan -Original Message- From: Beowulf On Behalf Of Christopher Samuel Sent: Tuesday, 23 July

Re: [Beowulf] Lustre on google cloud

2019-07-22 Thread Christopher Samuel
On 7/22/19 10:48 AM, Jonathan Aquilina wrote: I am looking at https://cloud.google.com/blog/products/storage-data-transfer/introducing-lustre-file-system-cloud-deployment-manager-scripts Amazon's done similar: https://aws.amazon.com/blogs/storage/building-an-hpc-cluster-with-aws-parallelclust

Re: [Beowulf] Lustre on google cloud

2019-07-22 Thread Chris Dagdigian
A lot of production HPC runs on cloud systems. AWS is big for this via their AWS Parallelcluster stack which does include lustre support via vfXT for lustre service although they are careful to caveat it as staging/scratch space not suitable for persistant storage.  AWS has some cool node typ

Re: [Beowulf] flatpack

2019-07-22 Thread Jonathan Engwall
I shoul add that a "flatpack" is any software you install and use which does not affect your os or env. Think turbotax. You insert the disk, do the math, print the form, eject. Audacity, for example, is a software I use. On Mon, Jul 22, 2019, 10:46 AM Jonathan Engwall < engwalljonathanther...@gmai

[Beowulf] Lustre on google cloud

2019-07-22 Thread Jonathan Aquilina
Hi Guys, I am looking at https://cloud.google.com/blog/products/storage-data-transfer/introducing-lustre-file-system-cloud-deployment-manager-scripts This basically allows you to deploy a lustre cluster on google cloud. In your HPC setups have you considered moving towards cloud based clusters?

Re: [Beowulf] flatpack

2019-07-22 Thread Jonathan Engwall
It is funny, I looked at their credits Codethink seems quite serious, Barron's thinks highly of Fastly, but Mythic Beast is a dynamic DNS and Scaleway is a public/private cloud that give a 500 pound starting credit. So, if you act now all these great titles can be yours. ___

Re: [Beowulf] flatpack

2019-07-22 Thread Christopher Samuel
On 7/21/19 7:30 PM, Jonathan Engwall wrote: Some distros will be glad to know Flatpack will load your software center with working downloads. Are you thinking of this as an alternative to container systems & tools like easybuild as a software delivery system for HPC systems? How widely supp