Hi P.R. wrote: > Im planning on building a small 20+node cluster, and I have some basic > questions. > We're planning on running 5-6 motherboards with quad-core amd 3.0GHz > phenoms, and 4GB of RAM per node. > Off the bat, does this sound like a reasonable setup >
I guess that fully depends on what you want to accomplish. If you want to use it as a proof of concept design or as a cluster for smaller tasks I think it looks reasonable. If you wanted to render IceAge4 with it, I think you need more power ;) > My first question is about node file&operating systems: > I'd like to go with a diskless setup, preferably using an NFS root for each > node. > However, based on some of the testing Ive done, running the nodes off of the > NFS share(s) has turned out to be rather slow & quirky. > Our master node will be running on a completely different hardware setup > than the slaves, so I *believe* it will make it more complicated & tedious > to setup&update the nfsroots for all of the nodes (since its not simply a > matter of 'cloning' the master's setup&config). > Is there any truth to this, am I way off? > 5-6 boxes off NFS root should not be a large burden to the server, as long as it has decent disk speeds (small RAID perhaps) and plenty of memory for caching (couple of GB should be sufficient to start with). Try tuning the NFS parameters to suit your needs. > Can anyone provide any general advice or feedback on how to best setup a > diskless node? Not really, we do that only during the installation phase. > > > The alternative that I was considering was using (4GB?) USB flash drives to > drive a full-blown,local OS install on each node... > Q: does anyone have experience running a node off of a usb flash drive? > If so, what are some of the pros/cons/issues associated with this type of > setup? > We do that only rarely for rescue setups as our nodes don't have CD drives and I think USB flash drives are pretty slow still. > > My next question(s) is regarding network setup. > Each motherboard has an integrated gigabit nic. > > Q: should we be running 2 gigabit NICs per motherboard instead of one? > Is there a 'rule-of-thumb' when it comes to sizing the network requirements? > (i.e.,'one NIC per 1-2 processor cores'...) > Again, that all depends on your workload and jobs. I think no-one can help you there unless you know what the workload will be. > > Also, we were planning on plugging EVERYTHING into one big (unmanaged) > gigabit switch. > However, I read somewhere on the net where another cluster was physically > separating NFS & MPI traffic on two separate gigabit switches. > Any thoughts as to whether we should implement two switches, or should we be > ok with only 1 switch? > Well again it depends what you need. I'd start first of with a single switch and see of NFS traffic is killing your MPI performance. On larger sites storage and intercommunication networks are often separated as the might interfere with each other too much, but it will boil down to the question how much money you have. 2 8-port GBit switches are cheap enough for testing, 2 1000+ Gbit switches (or Infiniband,...) are not ;) > > Notes: > The application we'll be running is NOAA's wavewatch3, in case anyone has > any experience with it. > It will utilize a fair amount of NFS traffic (each node must read a common > set of data at periodic intervals), > and I *believe* that the MPI traffic is not extremely heavy or constant > (i.e., nodes do large amounts of independent processing before sending > results back to master). > With the small number of machines I would go with 2 8 port switches and see what happens. No idea how wavewatch3 works and what it needs, sorry. > > Id appreciate any help or feedback anyone would be willing&able to offer... > I hope my reply already helps a little bit. Cheers Carsten _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
