On Wed, 2 Jul 2008, Carsten Aulbert wrote:
OK, we have 1342 nodes which act as servers as well as clients. Every node exports a single local directory and all other nodes can mount this.
Have you considered using a parallel file system ?
What we do now to optimize the available bandwidth and IOs is spread millions of files according to a hash algorithm to all nodes (multiple copies as well)
There have been many talks of improving performance by paying attention to the data locality on this very list. Are you not able to move the code to where the data is or move the data to where the code is ?
F.e. using a simple TCP connection (nc, rsh, rsync or even http) to transfer the file to the local disk before using it is probably more efficient than the way you use NFS is you deal with small files (as they have to be written to some local storage). The setup and tear-down costs of the NFS connection (automounter, mount, unmount) simply doesn't exist in this case; the transfer of data on the wire happens the same way. Or you could even get around the limitation of storing it locally by using a ramdisk to temporarily store the files (if you have the free memory...) - from what I understand they are read then used immediately and not needed again in a short time frame so it makes no sense to store them for longer, a perfect application for a tmpfs.
-- Bogdan Costescu IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany Phone: +49 6221 54 8869/8240, Fax: +49 6221 54 8868/8850 E-mail: [EMAIL PROTECTED] _______________________________________________ Beowulf mailing list, [email protected] To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
