I think I understand why people want the toolboxes - it makes coding easy. From
what I've seen people then stay with the "prototype" code and never move to a
compiled language such as C or Fortran. It's been a long, long time, but I did
all of my code prototyping for my PhD in Matlab and rewrote it in Fortran. I
was easily able to get a 10x speedup. I guess people don't like 10x
improvements in performance any more :)
An option for people is to use the Matlab compiler to build faster code. I've
never used it but the reports I've seen is that it works quite well. I'm not
sure about how the toolboxes work with it - whether you compile them as well or
they run as interpreted along with the compiled code.
Octave has a number of toolboxes. I'm not sure if it covers what various people
are doing, but they are out there.
For larger problems, what about using ScaleMP? You can get a very large SMP
system for running large problems. You may not need the extra CPUs, but they do
get you the extra memory. That's one of the neat things about ScaleMP. You buy
a couple of nodes with really fast CPUs and memory, and then buy the remaining
nodes with really slow processors (cheap) with lots of memory. This allows you
to tailor the SMP box to the application. Just be sure to run the code on the
fastest CPUs (numactl).
Thanks for the feedback.
Jeff
________________________________
From: Tim Cutts <t...@sanger.ac.uk>
To: Jeff Layton <layto...@att.net>
Cc: land...@scalableinformatics.com; Beowulf Mailing List <beowulf@beowulf.org>
Sent: Sunday, December 28, 2008 4:24:51 AM
Subject: Re: [Beowulf] Hadoop
On 27 Dec 2008, at 9:04 pm, Jeff Layton wrote:
> I hate to tangent (hijack?) this subject, but I'm curious about your
> class poll. Did the people who were interested in Matlab consider
> Octave?
I can't speak for Joe's class, but when I've asked Matlab users here
about using Octave instead, they're generally not interested. Partly
this is a somewhat irrational "it doesn't have Matlab on the cover"
thing, but largely it's the Matlab toolboxes they want access to.
A recurrent theme we find with anyone doing stuff with any similar
package, be it Matlab, Stata, R or whatever, is that they always hit
massive difficulties when they try to scale up their analysis to
larger problem sizes. If they come to me early in their project, I
will point this out in advance, and suggest that while such languages
are great for prototyping, they don't scale well. Usually, of course,
the warning is ignored (they didn't pay for this advice, so no need to
take it, right?) and six months down the line their stuff doesn't work
any more when they try to run it on large production datasets, and I
have to avoid the "I told you so" speech... :-)
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
