to generate a Universal FNN. FNNs don't really shine until you have 3 or 4
NICs/HCAs per compute node.
depends on costs. for instance, the marginal cost of a second IB port
on a nic seems to usually be fairly small. for instance, if you have
36 nodes, 3x24pt switches is pretty neat for 1 hop nonblocking.
two switches in a 1-level fabric would get 2 hops and 3:1 blocking.
if arranged in a triangle, 3x24 would get 1 hop 2:1, which might be
an interesting design point.
Though, as others have mentioned, IB switch latency is pretty darn small,
so latency would not be the primary reason to use FNNs with IB.
yeah, that's a good point - FNN is mainly about utilizing "zero-order"
switching when the node selects which link to use, and shows the biggest
advantage when it's slow or hard to do multi-level fabrics.
I wonder if anyone has built a FNN using IB... or for that matter, any
link technology
other than Ethernet?
I'm a little unclear on how routing works on IB - does a node have
something like an ethernet neighbor table that tracks which other nodes are
accessible through which port?
I think the real problem is that small IB switches have never really
gotten cheap, even now, in the same way ethernet has. or IB cables,
for that matter.
regards, mark hahn.
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
