On Thursday, April 8, 2010 2:14:11 PM Greg Lindahl wrote:
>> What are the approaches and experiences of people interconnecting >> clusters of more than128 compute nodes with QDR InfiniBand technology? >> Are people directly connecting to chassis-sized switches? Using multi-tiered >> approaches which combine 36-port leaf switches? > >I would expect everyone to use a chassis at that size, because it's cheaper >than having more cables. That was true on day 1 with IB, the only question is >"are the switch vendors charging too high of a price for big switches?" Hey Greg, I think my target is around 192 compute nodes, with room for a head node(s), and ports to a Lustre file server. So, 216 ports looks like a reasonable number to me (6 x 36). The price for an integrated chassis model solution should not exceed the price for a multi-tiered solution using 36-port (or some other switch smaller than 216) plus the cabling costs. Reliability and labor would also have to factored in with an advantage going to the chassis I assume based also on fewer cables? Looks like the chassis options are between $375 and $400 a port, while the 36 port options are running at about $175 to $200 a port (but you need more ports and cables). >> I am looking for some real world feedback before making a decision on >> architecture and vendor. > >Hopefully you're planning on benchmarking your own app -- both the >HCAs and the switch silicon have considerably different application- >dependent performance characteristics between QLogic and Mellanox >silicon. Yes, I assume that people would also recommend matching NIC and switch hardware. Thanks for your input ... rbw _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
