On Sun, 28 Sep 2008, Jon Forrest wrote:
There are two philosophies on where a compute node's OS and basic utilities should be located:
You forget a NFS-root setup, this doesn't require memory for the RAM disk on which you later mount NFS dirs.
In both cases it's important to remember to make any changes to this distribution rather than just using "pdsh" or "tentakel" to dynamically modify a compute node. This is so that the next time the compute node boots, it gets the uptodate distribution.
I prefer to look at the nodes as disposable, instead of "let's keep the node up as long as possible", so I usually don't modify a running system. Instead I modify the node "image" and reboot the nodes after the current jobs finish - this is easy to do when using a queueing system and is easy to hide from users when the typical jobs are longer than the reboot time.
However, on a modern multicore compute node this might just be a few percent of the total RAM on the node.
This also depends on how much of the distribution you keep as part of the node "image" and how you place the application software. It's often the case that the application software is distributed to the nodes from a cluster-wide FS, either from a directory holding software only or from the user's home dir; extending this to also include most of the libraries needed by the application software (f.e. fftw) means that the node "image" can be made very small without putting any part of the distribution on NFS (I know that some people totally dislike system utilities coming from a NFS mounted directory or depending on libs coming from one).
Approach #2 requires much less time when a node is installed, and a little less time when a node is booted.
I don't agree with you here as you probably have in mind a kickstart-based install for approach #1 running upon each node boot. I use for a long time a different approach - the node "image" is copied via rsync at boot time; the long waiting time for installing the RPMs and running whatever configuration scripts happens only once when the node "image" is prepared, the nodes only copy it - and it's a full copy only the first time they boot with a new disk, afterwards it's the rsync magic which makes it finish within seconds while making it look like a new disk :-)
-- Bogdan Costescu IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany Phone: +49 6221 54 8869/8240, Fax: +49 6221 54 8868/8850 E-mail: [EMAIL PROTECTED] _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
