Hi Greg,
Greg Lindahl wrote:
On Fri, Jan 30, 2009 at 07:07:34AM -0700, Michael H. Frese wrote:
>
Johnn Adams said "Facts are stubborn things," and there just aren't
enough of them in your example to determine whether bandwidth or latency
dominates communication time.
Mark asked for an example, not a research paper. And we were
And Michael gave you a good practical example. You can always find a
code that does something stupid like a 27-point (!) 3d stencil code
sending tiny messages. I know that there are such naive codes out there,
but reasonable stencil codes do exchange blocks that are not tiny,
easily transforming a message rate problem into a bandwidth problem. And
you could further transform the bandwidth problem into an overlap
problem with ghost cells.
network. In the Berkeley "logp" model, for example, processor
overhead and the "gap" betweeen messsages are fundamental parameters.
The InfiniPath chip has a tiny "o" and a negative "g". As a result,
Ok, my turn to bite :-) What is a negative "g" ?
it can send a lot more small messages than other interconnects, and
this number rises as you add more cores to a system (!).
Because the host library overhead (single core) is greater than the NIC
HW overhead. And that's when the host is doing nothing but sending. Add
some computation in there and the amount of messages a core will produce
is not that high. Does multiply it by the number of cores make it big
enough ? Maybe for a stencil code with tiny blocks where all the core
are very synchronized, ie they all send and receive at the same time and
wait for the data exchange before moving on.
In practice, stencil codes can start updating as soon as they receive a
border block, so not only you overlap communication and computation, but
not all cores end up communicating at the same time.
When the number of core will be one order of magnitude larger than today
(100s), then it will be a different discussion.
Patrick
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
