On Wed, Sep 9, 2009 at 2:12 PM, Bruno Coutinho <[email protected]> wrote: > Because final NFS server bandwidth will be the bandwidth of the most limited > device, be it disk, network interface or switch. > Even if you have a switch capable of fill line capacity for all 300 nodes, > you must put a insanely fast interface in your NFS server and a giant pool > of disks to have a decent bandwidth if all nodes access NFS at the same > time.
I'm thinking of having multiple 10GigE uplinks between the switch and the NFS server. The actual storage is planned to reside on a box of SAS disks. Approx 15 disks. THe NFS server is planned with at least two RAID cards with multiple SAS connections to the box. But that's just my planning. The question is do people have numbers. What I/O throughputs are your NFS devices giving? I want to get a feel for what my I/O performance envelope should be like. What kind of I/O gurrantees are available? Any vendors around want to comment? On the other hand just multiplying NFS clients by their peak bandwidth (300 x 1 GB) is an overkill. THat is a very unlikely situation. What are typical workloads like? Given x NFS mounts in a computational environment with a y GB uplink each what's the factor on the net loading of the central storage? Any back of the envelope numbers? > But depending on the way people run applications in your cluster, only a > small set of nodes will access NFS at the same time and a Ethernet 10Gb with > tens of disks will be enough. strace profiling shows that app1 has very little NFS I/O. App2 has about 10% runtime devoted to NFS I/O. Multiple seeks only. More reads than writes. (All this thanks to Jeff Laytons excellent strace analyzer and profiling help) -- Rahul _______________________________________________ Beowulf mailing list, [email protected] sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
