a dramatic breakthrough: 1.2 us, 25M msg/s. since we just happend to
or is this an example of message aggregation? heck, from the url
above, it might even be counting intra-box messages.
Nope, this is "normal" ping-pong for the new generation cards (connectx).
so if I do this:
start timer
send(other,small-message) recv(first,small-message)
recv(other,small-message) send(first,small-message)
stop timer
I'll actually see 2.4 us between the timer calls? if I understand,
aggregation would only help on a streaming test. in fact, this kind
of isolated RPC-like exchange is what I see most commonly.
Maybe a bit optimistic though, I'd expect closer to 1.5 in a back-to-back
config.
so for a small switch (24pt, say), how many hops to the internal fabric,
and they're, what, .2 us each?
also, does back-to-back work well? I can imagine some cases where
putting two dual-port cards in each node and creating a mesh might
work well.
also, I'm sorta amazed people keep selling (and presumably buying)
dual-port IB cards. doesn't that get quite expensive, switch-wise?
Not defending them but, It could possibly maybe be useful if you have a
stand-alone IB net for, say, storage or something else not mpi. Also, it's
not like they're that much more expensive than single port ones...
yeah, I can see PHB's buying redundant fabrics. I'd be more interested in
using the higher port-count for FNN or related topologies (assuming switches
are cheap, at least at some size...)
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
