found a copy of iperf on my system, so i ran it. show 0-10sec 10.6GBytes/sec 9.08Gbit/sec
so given that qperf seems to agree with iperf, i guess it's an interesting question now why, lustre lnet_selftest and IMB sendrecv seem throttled at 500MB/sec On Fri, Aug 4, 2017 at 1:21 PM, Michael Di Domenico <mdidomeni...@gmail.com> wrote: > thanks, most of that i've already tried. i don't have iperf, but i > did run qperf, which shows 1.0GB/sec and 16us latency between the two > hosts. but i'm not entirely sure i believe it > > the only thing that made a difference so far is changing the rx tx > rings up 8192 on the host. i went from 200-300MB/sec to 300-400MB/sec > consistently. but i'm still far short > > On Fri, Aug 4, 2017 at 12:47 PM, Alex Chekholko > <alex.chekho...@gmail.com> wrote: >> The ESNet pages are pretty good: >> https://fasterdata.es.net/host-tuning/ >> https://fasterdata.es.net/network-tuning/ >> >> Since you're on a local low-latency link, all the stuff about high-bandwidth >> high-latency links doesn't apply to you, but the host tuning parameters at >> least give you somewhere to look. >> >> I recommend iperf for testing before IMB, make sure you can get line rate >> throughput with an iperf test. >> >> On Fri, Aug 4, 2017 at 8:52 AM Michael Di Domenico <mdidomeni...@gmail.com> >> wrote: >>> >>> does anyone have any additional tuning tips beyond the random things >>> on the net for 10g ethernet tuning? i have two identical servers >>> (Quanta Q71L-4U) attached to a single cisco 10g switch, using >>> identical (Mellanox MNPH28B-XTC) cards. when i run mpi bandwidth >>> tests between the two cards, i top out at 200-300MB/sec using IMB >>> sendrecv. if i use lustre lnet selftests, i can crank the max to >>> 500MB/sec, but i can't seem to crest this. >>> >>> i've monkied around with the sysctl params and the cards are in x8 >>> slots at 2.5GT/s. i think they should be capable of driving at >>> ~900MB/sec, but i can't seem to get there. i know i'm missing >>> something... >>> _______________________________________________ >>> Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing >>> To change your subscription (digest mode or unsubscribe) visit >>> http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
