Mark Hahn wrote:
sure, and these are very fat nodes for which a fat interconnect is
appropriate for almost any workload that's not embarassing. but really
I wasn't suggesting that plain old Gb (bandwidth in particular) was
adequate for all possible clusters. I was questioning whether IB was a
panacea for small, cost-sensitive ones...
Cheap gigabit is fine for small cost sensitive clusters. You can get
cheap (not great, but ok) gigabit switches with 48 ports for under $700
today. They are not as fast as the higher cost ones from HP and others,
but they are great for inexpensive clusters.
At the small cluster side of things, the cost per core and cost per node
(fully burdened with switches, cables, OS, compilers, etc) is very
important. At these prices for some small clusters, the cost to add IB
is no longer completely prohibative. But, at the same time, the benefit
needs to outweigh the costs. I would argue that the more interesting
small clusters with IB probably won't be used for message passing, but
for storage using NFSoverRDMA to move large chunks of data back and
forth. There you get 5-8x better performance on your data xfer from
storage than you get with gigabit. For *some* workloads (life science
"database" analysis , large image processing, ...) this could be quite
important.
Most of the benchmark results (life science codes, chemistry codes,
engineering codes, ...) I have seen/worked on don't show a huge
difference between gigabit and IB until you get north of 32 cores. Of
course, that is 2-4 nodes these days ...
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: [EMAIL PROTECTED]
web : http://www.scalableinformatics.com
http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
