Re: [Beowulf] [External] RIP CentOS 8

2020-12-10 Thread Bill Abbott
I still haven't forgiven them for what they did to Sun. Bill On 12/10/20 2:44 PM, Jon Tegner wrote: On 12/10/20 10:55 AM, Jon Tegner wrote: What about https://linux.oracle.com/switch/centos/ Regards, /jon Possibly a good option - if I didn't trust Oracle even less than IBM.  I wonder if

Re: [Beowulf] First cluster in 20 years - questions about today

2020-02-02 Thread Bill Abbott
Mark, I'd start with OpenHPC on linux. They provide well-tested recipes with multiple software options and make the whole thing pretty easy. I'd use slurm for the scheduler (a single hetergeneous cluster) and the various other bundled OpenHPC components to manage nodes and so forth. We do

Re: [Beowulf] 10G and rsync

2020-01-02 Thread Bill Abbott
If you have no choice but to use single rsync then either set up an rsyncd server on the other end to bypass ssh or use something like hpn-ssh for performance. Bill On 1/2/20 10:52 AM, Bill Abbott wrote: > Fpsync and parsyncfp both do a great job with multiple rsyncs, although > you have

Re: [Beowulf] 10G and rsync

2020-01-02 Thread Bill Abbott
Fpsync and parsyncfp both do a great job with multiple rsyncs, although you have to be careful about --delete. The best performance for fewer, larger files, if it's an initial or one-time transfer, is bbcp with multiple streams. Also jack up the tcp send buffer and turn on jumbo frames. Bill

Re: [Beowulf] Oh.. IBM eats Red Hat

2018-10-29 Thread Bill Abbott
I like how a thoughtful piece on open source and freedom of choice ends with the phrase "You have no rights". Subtle. Bill On 10/29/18 7:44 PM, Douglas Eadline wrote: > > > In the sage words of Douglas Adams, "Don't Panic" > > My take here: > > > https://na01.safelinks.protection.outlook.com/?u

Re: [Beowulf] Intel Storm on the Horizon ?

2018-07-03 Thread Bill Abbott
I do plan on keeping an eye on 20B market cap HPC vendors. So far I've found exactly one. Bill On 07/03/2018 01:38 PM, C Bergström wrote: On Tue, Jul 3, 2018 at 10:35 PM, Douglas Eadline mailto:deadl...@eadline.org>> wrote: > It's an interesting theory, but just because Intel is off-s

Re: [Beowulf] Working for DUG, new thead

2018-06-13 Thread Bill Abbott
I thought it was for sysadmin, not developer. Disregard most of what I said. Bill On 06/13/2018 02:49 PM, Fred Youhanaie wrote: On 13/06/18 18:07, Jonathan Engwall wrote: John Hearne wrote:  > Stuart Midgley works for DUG?  They are currently  > recruiting for an HPC manager in London... Int

Re: [Beowulf] Working for DUG, new thead

2018-06-13 Thread Bill Abbott
One of my standard interview questions is to say ok, you start on Monday and you're placed in charge of a web/db server, tell me what you do your first week. What I want to hear is security, backups, log checking, monitoring, performance, functionality, etc., but most of all I want to know how

Re: [Beowulf] Working for DUG, new thead

2018-06-13 Thread Bill Abbott
they're in Australia, so they might use these terms differently. Prentice On 06/13/2018 01:53 PM, Bill Abbott wrote: linux, mostly On 06/13/2018 01:07 PM, Jonathan Engwall wrote: John Hearne wrote:  > Stuart Midgley works for DUG?  They are currently  > recruiting for an HPC manager

Re: [Beowulf] Working for DUG, new thead

2018-06-13 Thread Bill Abbott
linux, mostly On 06/13/2018 01:07 PM, Jonathan Engwall wrote: John Hearne wrote: > Stuart Midgley works for DUG?  They are currently > recruiting for an HPC manager in London... Interesting... Recruitment at DUG wants to call me about Low Level HPC. I have at least until 6pm. I am excited bu

Re: [Beowulf] Avoiding/mitigating fragmentation of systems by small jobs?

2018-06-08 Thread Bill Abbott
We set PriorityFavorSmall=NO and PriorityWeightJobSize to some appropriately large value in slurm.conf, which helps. We also used to limit the number of total jobs a single user could run to something like 30% of the cluster, so a user could run a single mpi job that takes all nodes, but could

Re: [Beowulf] Infiniband switch topology display

2018-01-17 Thread Bill Abbott
OSU's INAM is free and graphical. Bill On 1/16/18 12:18 PM, Alex Chekholko via Beowulf wrote: Hi John, My Mellanox knowledge is some years out of date, but Mellanox makes a proprietary and expensive monitoring tool with a GUI. I believe it has a 30-day trial mode, so you can install it and try