I think think that the web based interface always seems like a better
idea that it really is. We all have used the web, and we are comfortable
with web applications. There are also a number of successfull web based
scientific applications. But...
I see a web based submission system as an entry point. For a number of
years I have managed web based bioinformatics applications (generally on
SMP machines). I have found that they work best for more casual users.
As research becomes more sophisticated the web interface is usually
dropped in favor of a more powerful command line or alternately a
request is made for a wrappper to handle the application from a
simplified command line.
Mike Davis
[EMAIL PROTECTED] wrote:
rgb,
I was thinking more along the lines of a cluster - not a grid. This makes the
problem a bit easier. but as always the devil is in the details.
My thought was to have a some sort of file system in the cluster (pick your
poison) with the ability to mount it on the user's desktop (NFS, CIFS, or
whatever). Then the user would create their input files and put them on the
cluster storage (for the Windows users they just drag the directory to a new
drive). Then they enter a URL in the browser, log-in, tell the browser which
application they are using, where the files are located, how many processors
they want, and off the job goes. They could also use the same webpage to
track the jobs - how long has it been running, etc.? You could even put
diagnostics in there - if the nodes swap then tell the user to try more
processors
next time to speed up the job. You can also report back statistics if you want
to - how long the job took, how much IO was used, how much disk space was
used, etc. This might not tell the novice user too much, but it might tell the
admin something if there is a problem or it might tell the user or admin that
in the
future they might need a system with more IO, or a higher performing
interconnect.
The reason that I like the browser idea is that virtually every potential
cluster
user has used a browser before. In most cases they have even used webmail
or some sort of web site that requires logging in and entering data into a
field.
It's also very portable - it's web based.
Maybe, it's just me, but I find this idea much better than just hitting a button
that says "Run". This at least forces the user to think a bit about what they
are
doing instead of playing "Monkey hit the Button." There are examples of times
when people should have taken a few moments to think about what they were
doing instead of blindly just doing it. Plus, now it's not dependent on the
ISV's
putting a "Run" button in their GUI's (I'm amazed at the reluctance of ISV's to
change virtually anything in their code base - but that's another rant :) ).
Perhaps I'm not thinking as broadly as Rich. But I see a web-base solution as a
better idea than asking (or forcing) ISV's to put some new code in their
applications
to run on a cluster (BTW - in my experience some customers use the GUIs that
come with ISV codes and some don't.).
Jeff
On Thu, 12 Apr 2007, [EMAIL PROTECTED] wrote:
I really think the web interface is the way to go. This way you can submit jobs
from
any machine that has a browser (Linux, Windows, Mac, etc.).
Isn't that what gridware basically does already? Doesn't SGE provide a
web interface to resources (a cluster) it controls? Isn't that a
significant part of the point of the Globus project? The ATLAS grid
(IIRC) uses a grid interface, more or less, to provide just this sort of
layer isolation between cluster/grid resource and the user.
There are problems with this, of course. It is wonderful if the
grid/cluster already has a canned package installed, so that what the
user "submits" is a parametric dataset that tells the cluster how and
what to run with that package. BLAST etc work fine this way and there
exist cluster/grids architected just this way for this purpose, from
what I can tell via google. If you want to run YOUR source code on the
cluster from a different OS, however, well, that's a problem isn't it?
I think that there have been efforts to resolve even this -- ways of
submitting your source (or a functional binary) with build (or run)
instructions in a "package" -- I vaguely remember that ATLAS attempts to
implement a little horror called "pacman" for this purpose. I leave to
your imaginations the awesome mess of dealing with library requirements,
build incompatibilities, mistaken assumptions, and worse across
architectures especially ones likely for people who write in MS C++ (or
a C downshift thereof) and expect it the source to "just run" when
recompiled on a linux box.
Practically speaking, for source code based applications if the user has
a linux box (or even a canned vmware linux development environment they
can run as a windows appliance -- and there are many of them prebuilt
and available for free so this is no longer that crazy a solution on a
moderately powerful windows workstation -- and sufficient linux
expertise to work through builds thereupon, they can develop binaries or
build packages that they can submit to a cluster via a web interface
that hides all cluster detail. If not, then not.
Joe of course is building specific purpose clusters for many of his
clients and hence can successfully implement either canned software
solutions OR can manage the porting, building, preinstallation of the
client's software so that they can use it via a web-appliance interface.
Basically they purchase his expertise to do the code migration -- which
is again fine if the source is mature and unlikely to need a lot of
real-time tweaking and if they mostly want an appliance with which to
process a very large data space or parametric space a little at a time
(so "jobs" are parametric descriptions used to start up a task).
There are various other details associated with gridware and cluster
usage of this sort that make the idea "good" or "bad" per application.
If the application is bottlenecked by data access -- it processes huge
files, basically -- one can spend a lot of time loading data onto the
cluster via e.g. the web interface compared to a little time running the
application on the data, something that can perhaps be done more
smoothly and faster with a native shared disk implementation instead of
double hits on native disk on both ends plus a (probably slow) network
transfer. Accessing other resources -- GUI access to the program being
run, for example -- similarly depends strongly on having the right hooks
on both ends.
rgb
Jeff
Here is a proactive suggestion for keeping open source
ahead of Microsoft CCS:
1. I think CCS will appeal to small shops with no prior cluster
and no admin capability beyond a part time windows person.
2. such customers are the volume seats for a range of desktop
CAD/CAE tools.
3. Such ISVs will see potential of license growth, and will
likely choose to tie-in to the Microsoft message of ease-of-use.
A big feature here, in my view, is the one-button-job-launch.
This means, for Linux to have a position as the backend
compute cluster, we must have this one button job launch
capability. A Windows library must be available to
the ISV, to provide a job submission API to the batch
scheduler. With such a feature, the ISVs can be
persued to incoporate.
Ideally the job submission API is a kind of standard, so
the ISV does not see duplicate work versus the batch scheduler
used.
So,
a) we need a job submission API, and
b) we need the Windows library added to Linux batch schedulers.
(I'm not saying the scheduler runs on Windows, we just need
the submission/retrieve portion).
Does such exist already?
Thanks, Rich
Rich Altmaier, SGI
_______________________________________________
Beowulf mailing list, [EMAIL PROTECTED]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
_______________________________________________
Beowulf mailing list, [EMAIL PROTECTED]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
Robert G. Brown http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567 Fax: 919-660-2525 email:[EMAIL PROTECTED]
_______________________________________________
Beowulf mailing list, [EMAIL PROTECTED]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
_______________________________________________
Beowulf mailing list, [EMAIL PROTECTED]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
