On 5 Sep 2007, at 20:00, Eric Shook wrote:
Subject: Re: [Beowulf] Erlang as a language for Beowulf applications
To: "H.Vidal, Jr." <[EMAIL PROTECTED]>
I am engaged in similar explorations and I too have found the Erlang
language as a possible solution to scaling issues. I have been
experimenting with the language for almost a month so my evidence
below
is not by any means conclusive. With that said in my opinion the
language has a fatal flaw and that is data handling. According to the
documentation and my brief experience there are certain instances
where
the data being passed to and from processes / functions is copied and
not referenced. You can imagine the problems if you are manipulating a
multigig data set.
You are ahead of me, but, there are claims in the HiPE (High
Performance Erlang) papers (e.g. http://www.fantasi.se/jesper/
publications/TOPLAS06.pdf) that copying can be minimised with their
more sophisticated approach to memory management. I have not
confirmed this, though.
There are certain ways to handle bigger data sets
(mnesia, et al.) but I have not fully explored these.
If large data handling is not a problem for you I believe you will be
pleasantly surprised by Erlang. As you stated error handling and fault
tolerance are mature within the language and the lightweight processes
(300 bytes) are great for multicore machines, particularly
heterogeneous
machines.
I agree strongly. This has to be experienced. It is very easy to
write, and great fun.
In addition to these great features Erlang has built-in
functionality for distributed (e.g. across the network) communication.
The distributed communication is also platform independent (e.g.
Windows, Linux, etc) By combining fault-tolerance, distributed
communication, and lightweight processes one could easily enable an
application to adapt to multiple heterogeneous clusters, desktop
computers, and grid environments without modifications.
There are also interesting features to support 'live evolution' (i.e.
updating specific modules of the system without stopping it), which
may be handy for long running codes.
I feel this language is very promising. It will simply take an
individual or group to fully exploit the languages potential.
I haven't got any useful numbers yet, and I am still concerned that
raw Erlang performance may be too far short of its competition.
So, I am investigating using Erlang for distribution, fault
tolerance, management, and maybe live evolution. My approach is to
build a simple bridge to bits that need performance (which may
cripple the 'live evolution').
I would like to add a plug for Haskell, which is a statically typed
Functional language. It is beginning to show real promise on data-
parallel applications (likely still a year away, but much closer than
I had expected a year ago). Also, a guy (in the UK) has built some
neat Haskell optimisation transformations (for simple problems), with
the result that his *auto-magically transformed* Haskell code is
faster than the original C (and it is horrible to implement the
transformed algorithm in C).
Microsoft are supporting some of this effort, so it may come faster
than I think.
I'd like to understand what experience others may have of functional
programming languages on clusters, if any?
Garry
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
