On Wed, 25 Apr 2007, Amrik Singh wrote:
I agree that Jumbo Frames would not be a great help with the root file system but we hope to get a better performance from other NFS servers.
I don't quite follow you here: the link that was sent earlier had shown some kernel level autoconfig problem. If this is indeed still the case with newer kernels, this should only affect the root FS - before mounting the other NFS exports, you should have the chance to perform a proper initialization of the network, which would give you the possibility to use a larger MTU.
As all the machines on the same subnet have to be using the jumbo frames,
Why ? The MTU specifies a _maximum_ value; even when using 1500, not all packets are exactly 1500 bytes. The larger MTU _allows_ larger values, but doesn't _force_ them.
Even though NFS is extremely slow, copying files over scp is still very fast between a client and server.
If by this you mean that NFS is slower than scp, then this should be exactly the other way around, because scp needs CPU time for its encryption. I would have said that you have some network problems and use NFS over UDP which leads to high retransmission rates; scp would adapt better to the network problems due to using TCP... but you just mention that you tried switching between TCP and UDP.
We have tried all different ways to tune the NFS for a better performance (increasing NFS deamons on the servers, changing rsize & wsize, using TCP vs UDP, using async vs sync, noatime, timeo).
Could it be that you tried so hard to tune it that you just used too many settings that don't play well together in your particular situation ? I've found the NFS client and server in the kernels of the recent years to perform reasonably well in their default configuration; although they could be further optimized, a NFS transfer in these conditions would always beat a scp one.
initrd=bzImage ramdisk=40960
dhclient could be located in the ramdisk. Actually even a whole root FS could be located in the ramdisk, if there are not too many nodes booting at the same time and leading to UDP packet loss.
... and you already have a rather large ramdisk. Have you created it yourself (and therefore know what's in it and how to add some more) ?
-- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: [EMAIL PROTECTED] _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
