Robert G. Brown wrote:
On Thu, 2 Feb 2006, Bill Rankin wrote:
On Feb 2, 2006, at 8:04 AM, Robert G. Brown wrote:
What we do is use centos for servers (LAN/department servers, that also
serve the cluster nodes with e.g. home and project space). We use
FC-even revision numbers for desktops, cluster nodes, etc.
And for a slightly different view of other clusters at Duke :) - we
use Centos (currently 3.x, migrating to 4.x probably in the summer) on
the large central shared cluster. We have 500+ nodes right now and
are growing constantly. We currently support around two dozen
different research groups
We've used SuSE on our original Opteron cluster, but we've loaded Rocks
on the new Pentium-D nodes (not yet sure I made a good acquisition
decision there...
There are trade-offs in deciding about any distro. So far, both of
these have had niggling little issues but nothing to stop us from using
one or the other, and migrating to something else.
You should also point out that those nodes have been (over the same
period) Intel only, originally i386 only. We have had opterons since
FC1, and FC1 (badly) and FC2 were the only game in town to get opteron
support. Also back then Centos wasn't really flying smoothly yet, at
least on campus.
We tried FC1 for the opterons originally. They had come loaded with
SuSE 8 and there were issues there. FC1 didn't work badly for us... it
didn't work at all. SuSE 9 and upgrades (infrequent upgrades, stability
is good) have been pretty good.
Rocks has proven stable and predictable although there are some issues
we've seen in our scheduling and monitoring that probably reflect our
familiarity with other distros and systems.
As I said, we're migrating more to Centos at this point for nodes as
well, but it is a "no particular hurry" kind of migration as if we don't
fix roofs when it doesn't rain...
The main point is, using FCx isn't "crazy", it is just one thing one can
do, if/when there are reasons to do so (benefits) that outweigh the
hassles (costs). Same as any other system decision process, with a
similarly varied and highly nonlinear landscape that makes
second-guessing WHY somebody's costs and benefits are what they are (or
even just perceive them to be) a fruitless endeavor.
I've gotta agree with this assessment.
In other words "YMMV". There is a very wide range of distro options
these days, and none of them are properly "crazy". Well, maybe some are
(and no I won't indicate which ones as my asbestos suit is at the
cleaners:-), but FCx isn't one of them.
We are migrating to Solaris for our NFS servers. One area we're not
happy with in CentOS is the NFS service, and we've tuned it pretty
tightly. I'm really hoping Solaris can fix some of our performance issues.
gerry
on the campus with a mixed application pool off everything from
roll-yer-own MPI codes to commercial applications. Heck, we even
support Matlab to a very limited extent. :-)
For us, rolling out a new OS release is a major endeavor. Lots of
testing has to go on to verify that applications don't break on us and
that all the tool sets that we need are available. So for us, the
stability of Centos is a big attraction. Also one consideration for
third party apps is making sure
The stability, too, has a cost, though. A number of my apps wouldn't
compile at all with the GSL version that was standard in Centos 3 --
missing whole functions I needed. One then trades off rebuilding things
like GSL from current source (not a bad idea, but definitely regular
additional work IF any of your users need it) or using something like
FCx that updates libraries more regularly and is hence more likely to
have recently added functionality and performance and bugfix
improvements. Either way you're likely to have SOME work to do building
things on the side especially if you (like Mark H.) like to keep your
kernel bleeding edge current, or like to pick very specific high
performance snapshots of e.g. libc and freeze on them. Ultimately the
issue is which one is more work for you.
This issue has in the past been somewhat biased by virtue of the fact
that we run FCx on desktops, making it easi-er to do both nodes and
desktops from one distro (had to stabilize either one, right). But we
now do Centos and FCx either way, so again the point is moot.
Ultimately, with PXE/kickstart/yum and things like warewulf, nearly
ANYTHING like this becomes moot. The unavoidable work is in setting up
a cluster node and validating the distro you choose to install, and
hooking it up to a trustworthy update stream. At the end of a year,
that update stream is almost certain to be nearly frozen out anyway,
especially as far as cluster nodes behind a firewall are concerned. So
all that really matters is the compatibility stuff that you mentioned,
plus how MUCH work it is to validate any given choice and
build/rebuild/maintain stuff you need that is likely to be "prematurely"
frozen out in a conservative distro.
that they officially "support" the OS platform, in case we have to
deal with their technical support.
True, we do run into the cases where a user "must have" the
latest/greatest version of some library they use on their FC desktop,
or found in Debian and we try our best to accommodate them. But I
figure that we'll have that problem no matter what release of what OS
we run.
Exactly.
rgb
-bill
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
Gerry Creager -- [EMAIL PROTECTED]
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
