On Wed, Sep 9, 2009 at 10:40 AM, Rahul Nabar <rpna...@gmail.com> wrote: > Our new cluster aims to have around 300 compute nodes. I was wondering > what is the largest setup people have tested NFS with? Any tips or > comments? There seems no way for me to say if it will scale well or > not. > > I have been warned of performance hits but how bad will they be? > Infiniband is touted as a solution but the economics don't work out. > My question is this: > > Assume each of my compute nodes have gigabit ethernet AND I specify > the switch such that it can handle full line capacity on all ports. > Will there still be performance hits as I start adding compute nodes? > Why? Or is it unrealistic to configure a switching setup with full > line capacities on 300 ports? > > If not NFS then Lustre etc options do exist. But the more I read about > those the more I am convinced that those open another big can of > worms. Besides, if NFS would work I do not want to switch.
NFS itself doesn't have any hard limits and I have seen clusters well over a thousand nodes using it. With that said, you also need to consider your application and user requirements and the budget including administration costs as you architect your resource. As an aside note, generally the more specialized or non-standard the implementation, the more pressure you will put on administration costs. Keep in mind that the requirements of the system and budget need to define the architecture of the system. NFS is a good choice and can be suitable for systems much larger then 300 nodes. *BUT* that would depend on what you are doing with the cluster, application IO requirements, usage patterns, user needs, reliability/uptime goals, etc... Hope that helps. ;) -- Greg M. Kurtzer Chief Technology Officer HPC Systems Architect Infiscale, Inc. - http://www.infiscale.com _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
