Good point about swap. I often try an make the distinction that diskless booting (provisioning) does not require diskless nodes. That is, it is perfectly reasonable to use a centralized provisioning method and yet have HDD's on compute nodes -- if you need them. In the case where swap and local scratch space are needed this makes sense. (As long as the disks don't start collecting any cluster state on them)
-- Doug > Hi, > > Am 29.09.2008 um 12:27 schrieb Alan Ward: > >> Ram disks, definately. ;-) >> >> Afraid I'm still going with diskless nodes. You save >> >> 1. some money on the disks themselves >> 2. more money on solving disk failures >> > this highly depends on the used applications. Quantum chemistry code > like Gaussian03 or Molcas would run much slower, when all scratch > files would be written to NFS instead of a local disk. We even use > two or three stripped disks in the nodes. > > As the disk is in the node because of our applications anyway, I also > put the OS there. > > -- Reuti > >> >> 3. yet more money on cooling >> >> This may be specially important in a high-density rack situation, >> where if you get the disks out of the way each box can get more >> ventilators up front as well as in the rear. >> >> On the other hand, Windows Vista has had its uses(!), such as >> driving RAM prices down as demand expands ... >> >> Cheers, >> -Alan >> >> >> >> -----Original Message----- >> From: [EMAIL PROTECTED] on behalf of Jon Forrest >> Sent: Mon 9/29/2008 6:44 AM >> To: Beowulf Mailing List >> Subject: [Beowulf] Compute Node OS on Local Disk vs. Ram Disk >> >> There are two philosophies on where a compute node's >> OS and basic utilities should be located: >> >> 1) On a local harddrive >> >> 2) On a RAM disk >> >> I'd like to start a discussion on the positives >> and negatives of each approach. I'll throw out >> a few. >> >> Both approaches require that a compute node "distribution" >> be maintained on the frontend machine. In both cases >> it's important to remember to make any changes to this >> distribution rather than just using "pdsh" or "tentakel" >> to dynamically modify a compute node. This is so that the >> next time the compute node boots, it gets the uptodate >> distribution. Although the mechanism for maintaining >> the distribution varies in either approach, I consider >> this a push since one mechanism isn't inherently better >> than the other. >> >> Assuming the actual OS image is the same in both cases, >> #2 clearly requires more memory than #1. There are actually >> two approaches to #2 - a) where only the OS and other stuff >> necessary to boot the system are kept in memory and everything >> else is in an NFS-mounted file system, and b) where the whole OS >> installation is kept in memory. Depending on which approach >> is taken, the RAM-based installations can take hundreds of MB >> more than a local harddrive installation. However, on a modern >> multicore compute node this might just be a few percent of the >> total RAM on the node. >> >> Long ago not installing a local harddrive saved a considerable >> about of money but this isn't true anymore. Systems that need >> to page (or swap) will require a harddrive anyway since paging >> over the network isn't fast enough so very few compute nodes >> will be running diskless. >> >> Approach #2 requires much less time when a node is installed, >> and a little less time when a node is booted. >> >> What are some of your favorite issues, positive or negative, with >> each approach? >> >> Cordially, >> -- >> Jon Forrest >> Research Computing Support >> College of Chemistry >> 173 Tan Hall >> University of California Berkeley >> Berkeley, CA >> 94720-1460 >> 510-643-1032 >> [EMAIL PROTECTED] >> _______________________________________________ >> Beowulf mailing list, [email protected] >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> >> _______________________________________________ >> Beowulf mailing list, [email protected] >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ > Beowulf mailing list, [email protected] > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -- Doug _______________________________________________ Beowulf mailing list, [email protected] To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
