Matthieu, linking in a multi-threading enabled library to numpy, like
MKL, will provide speedup to only a limited set of operations and
scale only on when you limit yourself to the cores in a single smp-
like system. Star-P on the other hand, since it is MPI based, scales
on cluster and covers a much larger set of operations that can be
performed in parallel plus it provides a simple mechanism to do
embarrassingly parallel operations.
Ronnie
On Jun 29, 2007, at 1:46 PM, Matthieu Brucher wrote:
Hi,
Is there a comparison with parallel libraries that can be branched
on numpy like MKL ? (and IPP for random number ?)
Matthieu
2007/6/29, Ronnie Hoogerwerf <
[EMAIL PROTECTED]>:
I am an Application Engineer at Interactive Supercomputing and we
are rolling out a beta version of our Star-P product for Python. We
are actively looking for computationally intensive Python
application to port to Star-P. Star-P is a parallel application
development platform that allows users to tap into the power and
memory of supercomputers from the comfort of the favorite desktop
applications, in this case Python.
Star-P is capable of both fine-grained parallel computation and
embarrassingly parallel computation. The fine-grained mode of our
Star-P Python implementation has been modeled on the Python NumPy
package - for example:
x = starp.random.rand (20000,20000)
y = starp.linalg.inv(x)
instead of
x = numpy.random.rand(20000,20000)
y = numpy.linalg.inv(x)
Where the first couple of lines are executed on the Star-P parallel
server in full C/MPI mode and the last couple of lines are executed
on the desktop using Python.
The embarrassingly parallel mode is capable of executing any Python
module, although input and output parameters are currently limited
to NumPy arrays, scalars, and strings - for example:
y = starp.ppeval(mymodule.dosomething,x)
instead of
for i in range(0,n):
y[:,:,i] = mymodule.dosomething(x[:,i])
Where again in the former example the iterations are spread out
over the available CPUs (note the abstraction - user need not worry
regarding the number of CPUs) on the Star-P server using Python and
in the latter the looping is doing in serial on the client using
Python.
We are looking for real Python application that you would be
willing to share with us that we can port to Star-P. We want to
use this experience as a basis for further improvements and
development of our Python client.
Thanks,
Ronnie
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
