On Wed, Sep 9, 2009 at 3:38 PM, Greg Keller <g...@keller.net> wrote: >> > "It all depends" -- Anonymous Cluster expert
Thanks Greg. And I hate that anonymous expert. He's the bane of my current existence. I even get nightmares with his ghastly face in them. :) > I routinely run NFS with 300+ nodes, but "it all depends" on the > applications' IO profiles. 50% projected runtime is with an application with negligible reads and writes (VASP). The other 50% goes to an app. (DACAPO) which strace shows to be using 10% of its runtime devoted to I/O. Mostly seeks. More reads than writes. Multiple small reads and writes. All cores doing I/O not a central master core. >For example, Lot's of nodes reading and writing > different files in a generically staggered fashion, How do you enforce the staggering? Do people write staggered I/O codes themselves? Or can on alliviate this problem by scheduler settings? > Luster or eventually pNFS if things get ugly. But not all NFS servers are > created equal, and a solid purpose built appliance may handle loads a > general purpose linux NFS server won't. Disk array connected to generic Linux server? Or standalone Fileserver? Reccomendations? What exactly does a "solid purpose built appliance" offer that a Generic Linux server (well configured) connected to an array of disks does not offer? > The bottleneck is more likely the File-server's Nic and/or it's Back-end > storage performance. If the file-server is 1GbE attached then having a > strong network won't help NFS all that much. 10GbE attached will keep up > with a fair number of raided disks on the back-end. Load the NFS server up > with a lot of RAM and you could keep a lot of nodes happy if they are > reading a common set of files in parallel. Yup; I'm going for at least 24 GB RAM and twin 10 GigE cards connecting the file server to the switch. -- Rahul _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
