Re: [Beowulf] immersion [EXT]

The number of world wide DC’s that can accommodate 500kW/rack at any form of scale is very small. So this is an ever decreasing market, particularly for countries aligning with net zero commitments. So a market space is required. The view from some companies/orgs, is that the Hyperscalars will t

Re: [Beowulf] [External] head node abuse [EXT]

+1 to this. Relying on tech to solve issues like this tend to fall foul of new required software stacks and their requirements on head nodes, e.g. Nextflow, Cromwell etc. So a combined people process and education/assistance approach combined with some tech safty nets seems appropriate as long

Re: [Beowulf] Thoughts on git?

Thank you, The multi-language component of the link is very useful. One to pass on locally. Pete From: Beowulf on behalf of Alex Chekholko via Beowulf Reply-To: Alex Chekholko Date: Tuesday, December 19, 2017 at 4:45 PM To: John Hearns Cc: Beowulf Mailing List Subject: Re: [Beowulf] Thought

Re: [Beowulf] Thoughts on git?

+1 Also add in the potential for CI and CD as part of the hooks within git and you have a powerful method to ensure changes to your code are unlikely to impact your environment. By integrating with gitlab (etc) a GUI can also help your servicedesk accept proposals and have the updates then dep

Re: [Beowulf] slow jobs when run through queue

Show back of utilization and use patterns openly also removes admins from being “the Police”. Instead each user of the system can see who is requesting excessive memory, using inappropriate queues or just inefficient workloads at scale. This creates a self-Policing environment and certainly bo

Re: [Beowulf] Docker vs KVM paper by IBM

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/01/15 20:10, Joe Landman wrote: > > On 01/28/2015 03:03 PM, James Cuff wrote: >> "Large dependency radii" - love it! >> >> As one of those grumpy IT guys that control the spice, I for one >> embrace our new container overlords! Especially o

Re: [Beowulf] cloudy HPC?

On 10/02/14 12:09, John Hearns wrote: I wandered lonely as a cloud That floats on high o'er jobs and queues, When all at once I saw a crowd, A host, of golden CPUs; Beside the API, beneath the stacks, Fluttering and dancing in the racks. Now framed to be mounted on back of door in our DC Pete

Re: [Beowulf] Docker in HPC

On 28/11/13 22:44, Christopher Samuel wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 28/11/13 23:13, Bogdan Costescu wrote: > >> f.e. a single version of GROMACS > But then you run into the issue of different users who say "but I must > have GROMACS x.y.z", hence why we've already

Re: [Beowulf] Docker in HPC

Not at all. The restriction / affinity of jobs / process to a given core or core subset is very much in mind. Memory management is also potentially rather useful. In the case of most schedulers the memory used is obtained via a poll report. The enforcement of the memory limit has to date eithe

Re: [Beowulf] Docker in HPC

Hello Igor, We're on the same release stream and initially ran into some interesting memory resolution issues. The good news is that we have a pre-release patch in hand that is in testing (as we speak) if this proves to resolve the issues we ran into then I'm hopeful that the roll out will take

Re: [Beowulf] Docker in HPC

On 27/11/13 13:23, Lux, Jim (337C) wrote: From: John Hearns mailto:hear...@googlemail.com>> Date: Wednesday, November 27, 2013 4:35 AM To: "beowulf@beowulf.org " mailto:beowulf@beowulf.org>> Subject: Re: [Beowulf] Docker in HPC On 27 November 2013 12:29, Tim Cutt

Re: [Beowulf] Docker in HPC

On 27/11/13 11:45, John Hearns wrote: > Here is a Register article, which covers the same ground as Joe's post: > http://www.theregister.co.uk/2013/11/26/docker_spreads_to_more_linux_distros/ > " For instance, Docker could be used to run a database in one > container and an app server in another,

Re: [Beowulf] Restricting users from ssh into nodes

On 23/07/13 17:13, Chandler Wilkerson wrote: > We currently use a pam access setup like that: > > # cat /etc/security/access.conf > -:ALL EXCEPT admins root:ALL > > Then if users need access to the node while running jobs, we can do a > prolog/epilog that adds another line to allow in the user (the

Re: [Beowulf] Restricting users from ssh into nodes

sit http://www.beowulf.org/mailman/listinfo/beowulf -- --- Dr Peter Clapham, Informatics Systems Group The Wellcome Trust Sanger Institute, Cambs, CB10 1SA Tel: +44 (0)1223 834244 x 6972 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in Engl

Re: [Beowulf] hadoop

Lo all, I've pinged Ellis a mail with a couple of specific Q's but I wondered if anyone here has any experience with disco (http://discoproject.org/). I'm still in the early days of testing but would be interested to hear if anyone has an installation running at scale or in anger. Thanks Pete -

Re: [Beowulf] hadoop

On 27/11/2012 13:14, Vincent Diepeveen wrote: > Don't post something ridicioulous like that. > > That 64MB they use of course as with infiniband shipping a message of > 64MB wil get you the full bandwidth over the network > and keep number of messages down and infiniband doesn't work at your >

Re: [Beowulf] Degree

On 25/10/2012 14:26, Jonathan Dursi wrote: > On 25 Oct 9:22AM, Andrew Holway wrote: >>> Well, there was the propeller-top beanie and hazing when I first arrived >>> at graduate school, the secret physics handshake, the decoder ring, and >>> the wierd robes they made us wear in quantum mechanics "to

Re: [Beowulf] Degree

- yet those > British did do their PHD quite a lot of years ago... > > You're speaking for your own university and which department? > > Kind Regards, > Vincent > > On Oct 24, 2012, at 5:29 PM, Tim Cutts wrote: > >> On 24 Oct 2012, at 16:09, Peter Clapham wrote: >

Re: [Beowulf] Degree

On 24/10/2012 15:46, Vincent Diepeveen wrote: > On Oct 24, 2012, at 3:19 PM, Hearns, John wrote: > >> . >> >> Thing is, I need some kind of degree in this stuff to do the kind of >> work I really want to do. Especially in Germany, organisations >> involved in HPC usually strictly require an advance

Re: [Beowulf] clustering using xen virtualized machines

On the AWS ec2 side, we've been performing a range of tests including full genome sequencing pipelines across varying numbers of nodes and storage. The biggest challenge to date has been IO, particularly if the smaller image systems are used. Where jobs are highly cpu bound, little network (or

Re: [Beowulf] Configuration change monitoring

r unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf Dr Peter Clapham, Informatics System Group The Wellcome Trust Sanger Institute, Hinxton, Cambridge, CB10 1HH, UK -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in