If I understand correctly, 40GbE is 64/66 encoded.
Tom
On 3/15/2010 5:30 PM, Patrick Geoffray wrote:
On 3/15/2010 5:24 PM, richard.wa...@comcast.net wrote:
to best and worst case). It would be good to add Ethernet to the mix
(1Gb, 10Gb, and 40Gb) as well.
10 Gb Ethernet uses 8b/10b with a signal rate of 12.5 Gb/s, for a raw
bandwidth of 10 Gb/s. I don't know how 1Gb is encoded and 40 Gb/s is
still in draft. Last time I looked at 40 Gb/s, it was pretty much four
10 Gb links put together, so I would say 8b/10b with 50 Gb/s signal rate.
>There is a similar protocol efficiency at the IB or Ethernet level, but
>the MTU is large enough that it's much smaller compared to PCIe.
Would you estimate less than 1%, 2%, 4% ... ??
It depends on the packet size. For example, 14 Bytes Ethernet header on
1500 Bytes MTU, that's 1%. For Jumbo frames at 9000B MTU, it's much less
than that. I don't know the header size in IB, but with an MTU of 2K or
4K, it's negligible.
However, things are different for tiny packets. The minimum packet size
on Ethernet is 60 Bytes. The maximum packet rate (not coalesced !) is
14.88 Mpps on a 10GE link, counting everything (inter-packet gap, CRC,
etc). If you do the math, that's 14.88*60 = 892 MB/s on the link, or 684
MB/s if you remove the 14B Ethernet header (54% efficiency).
I don't think you can put all that on an Excel sheet :-)
Patrick
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
--------------------------------------------------------------------
Tom Ammon
Network Engineer
Office: 801.587.0976
Mobile: 801.674.9273
Center for High Performance Computing
University of Utah
http://www.chpc.utah.edu
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
