With your previous suggestions 8 months ago we bought a Tyan S4881 server
with 8 dual-core Opteron CPUs with 64GB RAM. Now we will buy new ones (2
more for the time being) and we will eventually planning to form a cluster
from these servers, which will have at most 8 boxes. Now, as you guess, the
critical question that we seek for the answer is how to interconnect these
fat nodes for max efficiency? Our parallel program is heavily loaded with
many small-size and large-size P2P communications.
what does that mean, exactly? you need to quantify your interconnect needs:
how often are programs waiting on file IO, for instance, and is that due to
the fileserver being slow or the net? similarly, how often are programs
waiting on communication among MPI ranks, for instance, and how much of that
weight is due to big messages (bandwidth)?
I feel that we need a
high performance network like Myrinet, IB, or Quadrics, not to waste 1us
latency and high bandwidth that we observe betw the 16 CPU cores. If anyone
I intuit (totally without rigor!) that fatter nodes do increase bandwidth
needs, but don't necessarily change the latency picture.
if I had three fat nodes, I'd probably try to use a dual-port IB card
in each, directly connected (no switches)...
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
