On Wednesday 28 March 2007, Mark Hahn wrote: > >> a dramatic breakthrough: 1.2 us, 25M msg/s. since we just happend to > >> > >> or is this an example of message aggregation? heck, from the url > >> above, it might even be counting intra-box messages. > > > > Nope, this is "normal" ping-pong for the new generation cards (connectx). > > so if I do this: > > start timer > send(other,small-message) recv(first,small-message) > recv(other,small-message) send(first,small-message) > stop timer > > I'll actually see 2.4 us between the timer calls? if I understand, > aggregation would only help on a streaming test. in fact, this kind > of isolated RPC-like exchange is what I see most commonly.
Assuming you could time it with any accuracy, yes. I've seen ib_write_lat figures of ~1. > > Maybe a bit optimistic though, I'd expect closer to 1.5 in a back-to-back > > config. > > so for a small switch (24pt, say), how many hops to the internal fabric, > and they're, what, .2 us each? Current switch chips are 24-port, so that means one hop (and yes, I think they are around .2 us each). > also, does back-to-back work well? I can imagine some cases where > putting two dual-port cards in each node and creating a mesh might > work well. That would require IB-to-IB routing on the hosts, I havn't heard of anyone doing that (don't think it's even implemented today). /Peter > >> also, I'm sorta amazed people keep selling (and presumably buying) > >> dual-port IB cards. doesn't that get quite expensive, switch-wise? > > > > Not defending them but, It could possibly maybe be useful if you have a > > stand-alone IB net for, say, storage or something else not mpi. Also, > > it's not like they're that much more expensive than single port ones... > > yeah, I can see PHB's buying redundant fabrics. I'd be more interested in > using the higher port-count for FNN or related topologies (assuming > switches are cheap, at least at some size...)
pgpQRCcD911R1.pgp
Description: PGP signature
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
