At 10:58 PM 5/25/2006, Patrick Geoffray wrote:
Hi Gilad,
Gilad Shainer wrote:
a) There's likely to be 10Gbps ethernet over ordinary cat 5e/6 cabling
soon. (Solarflare is one company working on it)
It is not a surprise, as you can run InfiniBand on cat 6 cables today.
There are several solution in the market that make it happened.
You can run anything on a shoe string between two yogurt cups today, but
it does not mean it's a good idea :-)
All Cat6 solutions have the same problems:
* overhead: KeyEye's transceivers for 10GigE (KX1001) adds 150ns. That's
0.3us for a point-to-point link if you assume it's per chip, twice that
with a switch in between. The transceiver for IB-SDR (KX1003) adds 200ns,
that's 0.4 us per link, 0.8 us with one hop.
I think that for any very high rate interconnect, latency starts to be a
significant factor (unless your messages are huge). 10G-T signals at
(IIRC) 800 Mbaud with a lot of bits/symbol (3-4, I think). However, the
real latency constraint is going to be the block size for the LDPC coding
which is 2048 bits. At 10 Gbps end to end throughput, those 2048bits will
take 204.8 nanoseconds, no matter how you deal with it. I imagine that the
decoder has some non-zero processing time.
Some of the competitive interconnects may do less coding (e.g. the cx4), in
which case the latency through the link could be shorter.
* power: between 3.5 and 3.7 watts per chip. That's more than the rest
of the NIC consumption. On a 256-port switch, that's almost an extra 1KW
to cool.
To a certain extent, speed costs power, because you need to get a certain
SNR to support the BER at the receiving end. 10x speed takes (to a first
order) 10x power (or, more accurately, the energy per bit remains roughly
constant, because the Eb/No determines the BER)
Figuring around 3-4W per port for 10G-T, what are the competitors (that can
support 10 Gbps) coming in at?
The Fujitsu CX4 transceiver is in the 2.1 Watt range, so we're not talking
order of magnitude differences. I wasn't able to tell in a quick glance
whether the transceiver provides any of the 8B/10B coding used, I assume it
does.
I suppose the real question is whether you even want to use Ethernet (in
any form) at all for these fast interconnects.
* length: 25-30 meters, not much more than CX4.
I think that the intent of the 802.3 standard for 10G-T is to support
55-100m on Cat 6, and no less than 100m on "Cat 7"
Granted, it would
certainly be a bit thinner than a 20 meters "firehose" CX4 cable, but it
does not help on the length side.
And the cable costs would be much lower with 10G-T. twisted pairs, for a
given performance, are going to be cheaper than twinax.
The only advantages I see are price of the cable (to be confirmed) and
look-like-cat5 warm feelings. Both are very important, but not for
technical reasons.
Patrick
Jim
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
