On 3/15/2010 5:24 PM, richard.wa...@comcast.net wrote:
to best and worst case). It would be good to add Ethernet to the mix (1Gb, 10Gb, and 40Gb) as well.
10 Gb Ethernet uses 8b/10b with a signal rate of 12.5 Gb/s, for a raw bandwidth of 10 Gb/s. I don't know how 1Gb is encoded and 40 Gb/s is still in draft. Last time I looked at 40 Gb/s, it was pretty much four 10 Gb links put together, so I would say 8b/10b with 50 Gb/s signal rate.
>There is a similar protocol efficiency at the IB or Ethernet level, but >the MTU is large enough that it's much smaller compared to PCIe. Would you estimate less than 1%, 2%, 4% ... ??
It depends on the packet size. For example, 14 Bytes Ethernet header on 1500 Bytes MTU, that's 1%. For Jumbo frames at 9000B MTU, it's much less than that. I don't know the header size in IB, but with an MTU of 2K or 4K, it's negligible.
However, things are different for tiny packets. The minimum packet size on Ethernet is 60 Bytes. The maximum packet rate (not coalesced !) is 14.88 Mpps on a 10GE link, counting everything (inter-packet gap, CRC, etc). If you do the math, that's 14.88*60 = 892 MB/s on the link, or 684 MB/s if you remove the 14B Ethernet header (54% efficiency).
I don't think you can put all that on an Excel sheet :-) Patrick _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
