I read through the beowulf archives for mentions of NFS + IB. I found nothing newer than 2012.
What are peoples current experience with NFS + IB? I'm looking at the options for smaller clusters with /home on NFS. I'll leave distributed filesystems for a separate discussion. The two leading options seem to be IPoIB and RDMA. Any comments on stability and performance assuming a fairly recent kernel like 3.13 or newer? I'd be especially interested in any particular settings like rsize, wsize, mtu (2044 for datagram mode), connected mode vs datagram mode, and similar tuning parameters. If you tuned for random I/O or bandwidth please mention it. I talked to one vendor that claims FDR has a larger than expected performance advantage over QDR for NFS. Can anyone verify or guess as to exactly why? Not sure if the IPoIB tuning is relevant for NFS + RDMA. I've read these links, if there are other useful links please let me know: https://www.kernel.org/doc/Documentation/filesystems/nfs/nfs-rdma.txt https://www.kernel.org/doc/Documentation/infiniband/ipoib.txt http://www.mellanox.com/related-docs/prod_software/Performance_Tuning_Guide_for_Mellanox_Network_Adapters.pdf I'm happy to summarise the results if people prefer sending info directly to me. _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
