On Wed, 23 Jul 2008, Ivan Oleynik wrote:
In principle, we have some experience in building and managing clusters, but with 40 node systems it would make sense to get a good cluster integrator to do the job. Can people share their recent experiences and recommend reliable vendors to deal with?
I suggest that you need a minimum of 16GB/node (2GB/core) and possibly 32GB/node (4GB/node).
You also should have a dedicated system for managing the cluster, a commodity PC with a lot of disk is adequate. That is the ideal place to host a DHCP server, tftp server (for for pxeboot), web/ftp/nfs server for installation and local mirror of the various repositories. This system should not be accessable by the users, just the admins.
You will want to set up IPMI on all of the nodes. You want to avoid touching the harware or even going into the machine room whenever practical.
Label both ends of each cable with source and destination. We have been using LSL-79 labels from wiremarkersplus.com. I am very fond of using velcro cable ties http://www.fastenation.com/class.php?id=9 for managing cables. I personally bought a spool of the 8" straps for use on my previous job and the folks at my current job bought a spool on my recommendation. Very useful.
We have also been buying custom ethernet cables in 1' increments from reynco.com. Very little price difference and ir radically reduced the rat's nest of cabling. Color coding by function/network is handy as well. Also, black ethernet cables inside of a black rack within a data center that is really well lit can lead to all sorts of headaches. Release your inner anti-goth and use brightly colored cables.
You will want to record the MAC address to physical location mapping sooner rather than later.
Install blanking panels to cover unused spaces in the rack. This will improve airflow.
Both Rocks and Warewulf seem to be excellent and have very friendly and supportive developers. Talking to them would probably be a good idea.
A much longer message than I planned, hopefully it is of use. -- Matt It's not what I know that counts. It's what I can remember in time to use. _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
