Re: [Beowulf] Cluster networking/OSs

Hi Trevor, I'm another BCCD developer. Since we target lower-end clusters, we don't support Infiniband, although as a Debian-based distribution it wouldn't be hard to install. Most of the software we support is pedagogical in nature - N-body simulations, numerical methods, etc. Our emphasis is on

Re: [Beowulf] Cluster networking/OSs

I’ll definitely check out BCCD. Thanks for the detailed response Jorg! Most of the clusters that I’ve worked with are generally used for large scale parallel jobs, I would be very interesting to learn more about your large cluster for running jobs like this. Is there any software your running a

Re: [Beowulf] Cluster networking/OSs

Hi Trevor, Not to toot my own horn here, but BCCD is designed with education in mind: http://bccd.net Aaron On Fri, May 8, 2015 at 3:30 PM, Trevor Gale wrote: > Hey Everyone, > > I’m fairly new to linux clusters and am trying to learn as much as I can > about specifically networking on cluster

Re: [Beowulf] Cluster networking/OSs

Hi Trevor, what kind of jobs do you want to run on the cluster? What I am after is: is it mainly number crunching, large parallel jobs, serial jobs, jobs which do a lot of disc IO, need much memory, visualisation. For example, I am not the 'master' of 8 clusters and most of them have differe

Re: [Beowulf] HPC in the cloud question

Hi, IBM Platform does provide IB for HPC with bare metal and cloudbursting, among other HPC services on the cloud. Detailed information including benchmarks can be found at http://www-03.ibm.com/systems/platformcomputing/products/cloudservice/ . Note that I work for IBM so I am obviously biased.

[Beowulf] Cluster networking/OSs

Hey Everyone, I’m fairly new to linux clusters and am trying to learn as much as I can about specifically networking on clusters and different operating systems run across clusters. Does anyone know any good resources to learn from? Thanks, Trevor __

Re: [Beowulf] HPC in the cloud question

Hi, Mike. We have been using StarCluster for some time, to deploy separate clusters in the cloud, per user. We update a custom CentOS 7 AMI that allows us to maintain binary compatibility with our Wharton HPCC system. This solution can be staff time intensive and/or require user training for depl

Re: [Beowulf] HPC in the cloud question

On 05/08/2015 10:04 AM, Jason Ingram wrote: Azure does offer InfiniBand based VM's, and CentOS is one of their six primary distributions. http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-endorsed-distributions/ I wish I had more to offer on the subject, I joined t

Re: [Beowulf] HPC in the cloud question

Azure does offer InfiniBand based VM's, and CentOS is one of their six primary distributions. http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-endorsed-distributions/ I wish I had more to offer on the subject, I joined this community as a personal choice to try

Re: [Beowulf] HPC in the cloud question

Mike, What are the characteristics of your cluster workloads? Are they tightly coupled jobs, or are they embarassingly parallel or serial jobs? I find it hard to believe that a virtualized, ethernet shared network infrastructure can compete with FDR IB for performance on tightly coupled jobs.

Re: [Beowulf] HPC in the cloud question

If you are on AWS start your eval with MIT Starcluster which is an amazing open source suite of python code that builds elastic HPC clusters on AWS with MPI, shared filesystem and all the stuff your users would be familiar with. Defaults to Grid Engine as the scheduler (super convenient for