Brandt Belson wrote: > Unfortunately I can't flatten the arrays. I'm writing a library where > the user supplies an inner product function for two generic objects, and > almost always the inner product function does large array > multiplications at some point. The library doesn't get to know about the > underlying arrays.
Now I'm confused -- if the user is providing the inner product implementation, how can you optimize that? Or are you trying to provide said user with an optimized "large array multiplication" that he/she can use? If so, then I'd post your implementation here, and folks can suggest improvements. If it's regular old element-wise multiplication: a*b (where a and b are numpy arrays) then you are right, numpy isn't using any fancy multi-core aware optimized package, so you should be able to make a faster version. You might try numexpr also -- it's pretty cool, though may not help for a single operation. It might give you some ideas, though. http://www.scipy.org/SciPyPackages/NumExpr -Chris > Thanks, > Brandt > > > > Message: 2 > Date: Fri, 10 Jun 2011 09:23:10 -0400 > From: Olivier Delalleau <[email protected] <mailto:[email protected]>> > Subject: Re: [Numpy-discussion] Using multiprocessing (shared memory) > with numpy array multiplication > To: Discussion of Numerical Python <[email protected] > <mailto:[email protected]>> > Message-ID: <[email protected] > <mailto:banlktikjppc90ye56t1mr%[email protected]>> > Content-Type: text/plain; charset="iso-8859-1" > > It may not work for you depending on your specific problem > constraints, but > if you could flatten the arrays, then it would be a dot, and you > could maybe > compute multiple such dot products by storing those flattened arrays > into a > matrix. > > -=- Olivier > > 2011/6/10 Brandt Belson <[email protected] > <mailto:[email protected]>> > > > Hi, > > Thanks for getting back to me. > > I'm doing element wise multiplication, basically innerProduct = > > numpy.sum(array1*array2) where array1 and array2 are, in general, > > multidimensional. I need to do many of these operations, and I'd > like to > > split up the tasks between the different cores. I'm not using > numpy.dot, if > > I'm not mistaken I don't think that would do what I need. > > Thanks again, > > Brandt > > > > > > Message: 1 > >> Date: Thu, 09 Jun 2011 13:11:40 -0700 > >> From: Christopher Barker <[email protected] > <mailto:[email protected]>> > >> Subject: Re: [Numpy-discussion] Using multiprocessing (shared > memory) > >> with numpy array multiplication > >> To: Discussion of Numerical Python <[email protected] > <mailto:[email protected]>> > >> Message-ID: <[email protected] > <mailto:[email protected]>> > >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >> > >> Not much time, here, but since you got no replies earlier: > >> > >> > >> > > I'm parallelizing some code I've written using the built in > >> > multiprocessing > >> > > module. In my application, I need to multiply many > large arrays > >> > together > >> > >> is the matrix multiplication, or element-wise? If matrix, then numpy > >> should be using LAPACK, which, depending on how its built, could be > >> using all your cores already. This is heavily dependent on your your > >> numpy (really the LAPACK it uses0 is built. > >> > >> > > and > >> > > sum the resulting product arrays (inner products). > >> > >> are you using numpy.dot() for that? If so, then the above applies to > >> that as well. > >> > >> I know I could look at your code to answer these questions, but I > >> thought this might help. > >> > >> -Chris > >> > >> > >> > >> > >> > >> -- > >> Christopher Barker, Ph.D. > >> Oceanographer > >> > >> Emergency Response Division > >> NOAA/NOS/OR&R (206) 526-6959 > <tel:%28206%29%20526-6959> voice > >> 7600 Sand Point Way NE (206) 526-6329 > <tel:%28206%29%20526-6329> fax > >> Seattle, WA 98115 (206) 526-6317 > <tel:%28206%29%20526-6317> main reception > >> > >> [email protected] <mailto:[email protected]> > >> > > > > ------------------------------------------------------------------------ > > _______________________________________________ > NumPy-Discussion mailing list > [email protected] > http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [email protected] _______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
