Important paragraph: "Some larger players in the HPC arena have begun to provide rich support for high-performance parallel file systems as a complete alternative to HDFS. IBM's GPFS file system has a file placement optimization (FPO) capability that allows GPFS to act as a drop-in replacement for HDFS, and Intel was selling native Lustre support before they sold IDH to Cloudera."
And Intel still are but are bundling their Lustre <-> Hadoop connector with their commercially supported Lustre product "IEEL". It's not OSS however. They support Cloudera (obviously as they just sunk $740m) and Apache Hadoop. There is also support for SLURM as a scheduler however I don't have a nice link. These bits should mean that Hadoop can play nicely as a "normal" HPC application. On 17 May 2014 22:46, Kilian Cavalotti <kilian.cavalotti.w...@gmail.com>wrote: > Great write-up by Glenn Lockwood about the state of Hadoop in HPC. It > pretty much nails it, and offers an nice overview of the current > ongoing efforts to make it relevant in that field. > > > http://glennklockwood.blogspot.com/2014/05/hadoops-uncomfortable-fit-in-hpc.html > > Most spot on thing I've read in a while. Thanks Glenn. > Cheers, > -- > Kilian > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf >
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
