Joe, et al,
Unfortunately, we find, time and time again, when Microsoft comes in to
tell us how good they're gonna do something, there's a catch.
Here, it remains simple to me: I can tweak the OS if I need to, and I
can tweak the apps. With a Windows solution fo rmy cluster environment,
I certainly can't tweak the OS. I'm half-way to dead in the water.
In general, we integrate our own hardware. If I found the API was too
complex, I'd complain mightily, but actually, I tend to pick hardware I
can already talk to, and API's I'm already familiar with to program to.
At this point, Microsoft is claimint to save me from myself. Hmmm.
I'm reminded of the time IBM decided... and advertised... their intent
to "legitimize" the minicomputer industry. If I recall correctly, the
Data General ad in response included the comment, "The bastards say,
'Welcome'." That's sort of what I feel here.
I'm reminded that Microsoft made attempts to wander into this community
several years ago. I further recall being simply told that there was no
direction for Linux, no core of paid developers, and that the HPC work I
was doing couldn't expect to continue using that paradigm. Simply, I
should migrate my applications to Windows (NT4, at that time) and wait
for the Microsoft HPC tool.
I'm not sure I'm ready for Microsoft to define my timeline and my code
requirements.
I'll stand by my position. The bulk of the Microsoft announcement is
marketing, and demonstrates a less than stellar acknowledgment of the
HPC environment they're trying to break into. If y'all think your
corporate wagon's hitched to their star, that's great. I'll watch and
learn. I'm not completely wedded to Linux, but I'mnot likely to change
until it does something similar to a series of data losses eventually
identified as registry corruption which encouraged me to make the swap
to Linux. As long as the problems I have continue to be hardware
related (think, statistical failure rates) and not OS definable, I doubt
I'll seriously investigate them.
gerry
Joe Landman wrote:
(sorry in advance for the length)
Gerry Creager N5JXS wrote:
Ooh! ISO-9002 Buzzword Compliant marketing.
Hmmm.... not astroturfing here. I opine on http://scalability.org/?p=69
. A few others have linked to this, so we are getting some traffic.
Specifically, I am of the opinion that the sentence "but until now it
has been too expensive and too difficult for many people to use
effectively" is factually wrong. The reasoning is very simple, and
borne out by existing data.
If HPC has been both too expensive AND too difficult to use, then why is
it as a market growing at 20+% per year? Moreover, as IDC points out
that Linux clusters are driving the growth in the HPC market, with this
segment currently at about 1/2 of the $9B market and growing at 60+% per
year for the past 3 years ...
Short version: it is neither too expensive, nor too hard, as people are
doing it effectively now, and have been for the last 5+ years, with the
growth showing up on the radar 3 years ago.
If the statement of too expensive could be applied to Linux clusters,
then this fails to explain the observation of the growth. Last I
checked, real measurement trumps hypothesis. These aren't windows
clusters that are growing, these are Linux clusters. These aren't unix
boxen. They are Linux clusters. There must be a reason for this.
If the statement of too hard could be applied to the market, one would
need to ask exactly what people were buying that was not too hard which
is generating all that growth. Since we know the answer (linux
clusters), they must not be too hard to use. The systems we put
together for our customers who don't care what is under the hood looks a
great deal like a large windows disk (or disks) and a web page. Those
who care about the details prefer the command line.
All this said, and not to disagree with Doug Eadline and others on the
technical details, I do think Microsoft has something to offer here, but
I think they need to work within the existing community, and not dismiss
it out of hand. The latter is the sense I am getting out of the
marketing. The folks from Microsoft I have spoken with, Kyril and
Patrick, seem to be quite interested in doing the right thing, though
with a decidedly Microsoft spin. The problem I see is that the spin and
some of the core assumptions are, IMO, incorrect. One expects marketing
to be so. Building a go-to-market strategy upon the basis of core
assumptions that are not in line with the market reality is dangerous,
even for an infinitely deep pocketed corporation. All I suggest is
appropriate debunking of the marketing, and drilling into the
technological core of what they are going to market.
They do have a number of very hard hills to climb, specifically pricing
compared to competitors, technological feature lists, interoperability,
security, and stability. Most of these are going to work against it.
It would be unwise to count them out of the game though. Anyone
remember or still use Lotus 123? Wordperfect? May take them a while,
and they are persistent. With very deep pockets, lots of patience, and
the ability to purchase talent.
Linux was able to effectively kill Unix by presenting a single API to
write to, a simple stack to deal with, a much larger potential installed
base, a lower cost of acquisition. Microsoft has learned from this.
Assume that this is their direction. The arguments they presented to me
involved driving a wedge between various linux distros, and painting the
Linux scene in a similar manner. Their MPI argument (to many stacks)
was not a good one, as the same problem exists on windows. But the
point is one that I and many others have complained about at some point
in time or the other. You have different MPI stacks which are binary
incompatible. Which means if the PathScale folks came out with a new
hardware device to accelerate networking for folks like LSTC, then the
LSTC folks have to relink their app against the new stack. Which is
exactly what happened. While some folks here defend this, I want to
note that end users don't give a rip about that. They want the new
fangled hardware to work. Right away. Without a rebuild of the app.
So do the vendors.
ISVs don't want a rotating collection of MPI stacks, but one to work
with. One API. Each app now needs to decide how many MPI stacks to
support on each ABI. You have IA64 with several, AMD64 with several,
ia32 with several, ... Each stack adds cost/complexity to them. Again,
this is something I have argued for a while. The ISVs as well. They
don't want to support Joe's MPI stack, they want one MPI stack per ABI
(even better would be one MPI stack and one ABI, but we are not there
yet). MPICH runs pretty much everywhere. LAM (which I like a little
better) runs fewer places (never been able to get it to work the right
way under Cygwin). I can live with MPICH. The problem though is that
we have to relink the application for each new networking advance. And
customers don't like that.
What Microsoft will do is to take away as much of this as they can. I
haven't seen it yet, but I believe they will offer MPICH as a DLL, so if
PathScale wants to work along side some other device, you can select
this at runtime, and just have it work. This is a nice idea.
Schedulers are another area, but my impression from speaking with them
is that they haven't looked at the market carefully enough yet, or some
non-business reasons got in the way of them exploring whats out there.
The idea is that they have a good story in some parts, and a very weak
story in others. Assume they will improve the weak points. It would
make more sense to engage them so that they improve the weak points
along reasonable lines. I would rather have them fit in, than try to
overtake, as the latter will just piss off the customers when they
realize what they want to do is not possible with their shiny new WCC,
or they suffer far worse performance than the folks with the Linux
cluster due to all those corporate mandated firewalls and copies of
Norton running.
Joe
--
Gerry Creager -- [EMAIL PROTECTED]
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
