Actuall I run it in production and I'm not a kernel hacker. We currently have 6 OSS's with software raid5 to 6 internal SATA disks. We see about 190MB/s per OSS out of the disks and around 150MB/s via the dual network interfaces.
I can't think of any benchmark you care to mention that a single lustre OSS/MDS won't outperform NFS. Especially, if you configure your systems to use both NIC's (most motherboards now come with dual interfaces) and I don't mean trunking the ports. Just configure portals to know that it can speak to the OSS's via both nics and it will handle the rest for you. Lustre's meta data performance is WAY better than NFS. I'd almost say its WAY better than any global FS I've played with. Certainly, you have to use Lustre kernels... all we do is run Centos as our clients/servers and then we just grab the pre-build/supported kernels from CFS. Its all pretty easy. The current Lustre V1.4 is very very nice and we have found it to be very robust. Nearly all the problems we experience turn out to be flakey hardware or kernel issues. Not Lustre at all. You can also checkout the FUSE implementation of a Lustre client I posted to CFS's website a few weeks back https://mail.clusterfs.com/wikis/lustre/fuse while it needs a LOT of work to give decent performance, it does work. Oh, and if someone ports liblustre to macosx, I could also run it on my mac :) Stu.
How much do you use Lustre? Yes, you can get that bandwidth, but if you code doesn't do large-streaming I/O, you performance will be worse than NFS. Also, I would like to hear someone speakup that uses Lustre in a PRODUCTION environment that doesn't have a kernel hacker on staff. Also, Lustre metadata doesn't scale (yet). You can add another server, but that won't improve the metadata. Using Lustre also requires you to re-patch your kernel every security update, then get the bugs out again. Lustre is the right answer for some, but if you aren't going to have that many compute nodes. It doesn't sound like it here. Craig
-- Dr Stuart Midgley [EMAIL PROTECTED] _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
