Date: Mon, 11 Jan 2010 09:58:24 -0500
From: Michael Di Domenico <mdidomeni...@gmail.com>
Subject: Re: [Beowulf] cisco networking
To: Glen Dosey <dos...@r-networks.net>
Cc: Beowulf@beowulf.org
Message-ID:
<e75d22a91001110658k6b5e30eaq83e8a83080cc...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
On Sun, Jan 10, 2010 at 1:29 AM, Glen Dosey <dos...@r-networks.net>
wrote:
You can have multiple ether-channel links between 2 switches. The
limitation is that if they are in the same layer 2 broadcast domain
(VLAN) only 1 will be active and STP will block the other.
If you place each switch in a separate network you could use equal
cost
load balancing across multiple point to point SVI's on ether-channel
links. You'll increase bandwidth at the cost of a little additional
latency. Of course at the bandwidths you are talking about (160
Gbit/s
or more ) I have no idea how it would really work. Backplane
bandwidth,
hashing algorithms and all sorts of other factor come into play and
could cause it to fail completely. I'd love to try it out but I can't
help but think there is probably a better architectural solution.
What are you trying to do ?
The attempt is really to get a bunch of devices which only have
ethernet 1Gbps connectivity, to talk to a bunch of other machines with
Infiniband Connectivity. (Ie, compute nodes to storage). But we want
the aggregate bandwidth to be over 5GBps.
I was trying to see if there was a simple solution using some of the
existing stuff we have, without having to get overly creative with the
network or purchase 10G Ethernet to IB routers...
Looks like IB routers is the only solution that really works for me...
------------------------------
Remember that you can actually use nodes to route IP between GbE and
IB, just don't expect them to run at IB wirespeed. This can at least
save you from paying a premium for proprietary hardware in order to
test functionality. Last I tried 3-4 Gbit was about as fast as IP
over IB would get through the Interface presumably due to IP
overhead... but that was long ago so if you try it I'd love to hear
your results.
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
