Re: [Numpy-discussion] Efficient square distance computation

2013-10-10 Thread Ke Sun
e.g. with openblas I could also do this size in about 100 seconds wall time > (20 minutes cpu time). I recompiled numpy+scipy with intel compiler and MKL. Now it takes around 2 MINUTES on my machine. Horay! Thanks! Best, Ke Sun ___ NumPy-Discussion ma

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Ke Sun
On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote: > Hi, > > On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun wrote: > > Dear all, > > > > I have written the following function to compute the square distances of a > > large > > matrix (each sample a ro

[Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Ke Sun
utput should be a 70,000x70,000 matrix. The program runs really slow (16 hours for 1/3 progress). And it eats 36G memory (fortunately I have enough). Could you give some insights on how to modify the code to be efficient and to eat less memory? thanks, Ke Sun def dist2_large( data ): import