application environments. Contrary to the detractors of the technologies
comments, the TOE/RDMA card *did* provide fairly significant performance
delta for real apps running MPI over gigabit ethernet.
well, let's not conflate these things: TOE and RDMA are not intrinsically
related...
I won't talk on the business side of them. We did see 4x better (wallclock)
time on customers StarCD calculations being run over the TOE/RDMA engine than
over the pure gigabit path.
do you have a sense for whether this was due to RDMA giving you lower latency
by bypassing the stack (either host or the nic-toe stack)? I don't think
I've ever heard of TOE itself providing any latency benefit.
also the more appropriate comparison is not a normal mpi-tcp-ip stack, but
rather something like GAMMA or OpenMX. I don't suppose you had a chance
to test those, did you?
Customer was pleased until they realized that
the design of this card effectively killed their maximum memory size (3.1 GB
vs the 4 GB installed). This was the iWarp bit.
I'm guessing that techniques from the IB world would ameliorate this...
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
