Hello all, Since I haven't heard back, I decided to simplify my problem even more. I'm attaching two files which demonstrate the problem very clearly. My original message is below. The script can be run with "python shared_mem.py". Thanks, Brandt
> > Hello, > > I'm parallelizing some code I've written using the built in > multiprocessing > > module. In my application, I need to multiply many large arrays together > > and > > sum the resulting product arrays (inner products). I noticed that when I > > parallelized this with myPool.map(...) with 8 processes (on an 8-core > > machine), the code was actually about an order of magnitude slower. I > > realized that this only happens when I use the numpy array > multiplication. > > I > > defined my own array multiplication and summation. As expected, this is > > much > > slower than the numpy versions, but when I parallelized it I saw the > > roughly > > 8x speedup I expected. So something about numpy array multiplication > > prevented me from speeding it up with multiprocessing. > > > > Is there a way to speed up numpy array multiplication with > multiprocessing? > > I noticed that numpy's SVD uses all of the available cores on its own, > does > > numpy array multiplication do something similar already? > > > > I'm copying the code which can reproduce and summarize these results. > > Simply > > run: "python shared_mem.py" with myutil.py in the same directory. > > > > Thanks, > > Brandt >
shared_mem.py
Description: Binary data
myutil.py
Description: Binary data
_______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
