Re: [Numpy-discussion] about SIMD (SSE2 & SSE3)

2010-11-06 Thread David Cournapeau
On Sat, Nov 6, 2010 at 9:51 PM, qihua wu wrote: > I used the following command to install the numpy to enable the SSE3 > numpy-1.5.1rc1-win32-superpack-python3.1.exe /arch sse3 The whole point of the super pack installer is to install the most optimized one possible on your machine. So you should

Re: [Numpy-discussion] about SIMD (SSE2 & SSE3)

2010-11-06 Thread David Warde-Farley
On 2010-11-06, at 7:46 PM, qihua wu wrote: > day 1,2,3 have the non-promoted sales, day 4 have the promoted sales, day > 5,6,7 have the non-promted sales, the output for day 1~7 are all non-promoted > sales. During the process, we might need to sum all the data for day 1~7, is > this what you c

Re: [Numpy-discussion] about SIMD (SSE2 & SSE3)

2010-11-06 Thread qihua wu
Thank David, the java program takes 3 hours to read data, after read the data into memory, it takes 4 hours to process/calculate somthing on all these data. The data is the sale data which contains both promoted sale and non-promoted sale, the program needs to predict the non-promoted sale: so in

Re: [Numpy-discussion] Optimize Floyd-Wallshall algorithm with Numpy

2010-11-06 Thread John Salvatier
The difference is that dis[k,:] eliminates the first dimension since you are using a single number as an index, but dis[k:k+1,:] does not eliminate that dimension. On Sat, Nov 6, 2010 at 1:24 PM, wrote: > On Sat, Nov 6, 2010 at 4:14 PM, K. Sun wrote: >> Thanks a lot. It works! I modify the code

Re: [Numpy-discussion] Optimize Floyd-Wallshall algorithm with Numpy

2010-11-06 Thread josef . pktd
On Sat, Nov 6, 2010 at 4:14 PM, K. Sun wrote: > Thanks a lot. It works! I modify the code as follows and it runs > at fast as matlab. By numpy's convention, the input and output > are all ndarrays. 'route' has to be a (1xN) matrix to produce a > square matrix in 'route + route.T'. If you read my

Re: [Numpy-discussion] Optimize Floyd-Wallshall algorithm with Numpy

2010-11-06 Thread K. Sun
Thanks a lot. It works! I modify the code as follows and it runs at fast as matlab. By numpy's convention, the input and output are all ndarrays. 'route' has to be a (1xN) matrix to produce a square matrix in 'route + route.T'. def floyd( dis ): '''Floyd-Wallshall algorithm for shortest path

Re: [Numpy-discussion] about SIMD (SSE2 & SSE3)

2010-11-06 Thread David Warde-Farley
On 2010-11-06, at 8:51 AM, qihua wu wrote: > I used the following command to install the numpy to enable the SSE3 > numpy-1.5.1rc1-win32-superpack-python3.1.exe /arch sse3 > > Then how can I know whether numpy is running with SSE or not? As far as I know, the only thing that uses SSE/SSE2/SSE3 w

Re: [Numpy-discussion] Optimize Floyd-Wallshall algorithm with Numpy

2010-11-06 Thread josef . pktd
On Sat, Nov 6, 2010 at 3:28 PM, K. Sun wrote: > Hello, > > I wrote the following code with numpy to implement the Floyd-Wallshall > algorithm to compute the pair-wise shortest path in a undirected weighted > graph. It is really slow when N ~ 10k, while the same implementation in > matlab is much f

[Numpy-discussion] Optimize Floyd-Wallshall algorithm with Numpy

2010-11-06 Thread K. Sun
Hello, I wrote the following code with numpy to implement the Floyd-Wallshall algorithm to compute the pair-wise shortest path in a undirected weighted graph. It is really slow when N ~ 10k, while the same implementation in matlab is much faster. I am sorry I don't want to run it again to present

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread josef . pktd
On Sat, Nov 6, 2010 at 11:52 AM, Damien Moore wrote: > In reply to my own question, the trivial, but massively inefficient solution > is: > > s=StringIO.StringIO('q1,2\nq3,4') > a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda s:float(s[1:])}) > a1 = numpy.array(a.tolist()) > > But what I r

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Lluís
Sorry, I got it wrong and ignored the StringIO part. Lluis Lluís writes: > Damien Moore writes: [...] >> import numpy, StringIO >> s=StringIO.StringIO('q1,2\nq3,4') >> a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda s:float(s[1:])}) >> s=StringIO.StringIO('q1,2\nq3,4') >> b=numpy.genf

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Lluís
Damien Moore writes: > Hi List, > I'm trying to import csv data as a numpy array using genfromtxt. The csv file > contains mixed data, some floating point, others string codes and dates that I > want to convert to floating point. The strange thing is that when I use the ' > converters' argument to

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Damien Moore
In reply to my own question, the trivial, but massively inefficient solution is: s=StringIO.StringIO('q1,2\nq3,4') a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda s:float(s[1:])}) a1 = numpy.array(a.tolist()) But what I really want to do is have genfromtxt do the conversion for me. Specif

[Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Damien Moore
Hi List, I'm trying to import csv data as a numpy array using genfromtxt. The csv file contains mixed data, some floating point, others string codes and dates that I want to convert to floating point. The strange thing is that when I use the 'converters' argument to convert a subset of the columns

[Numpy-discussion] about SIMD (SSE2 & SSE3)

2010-11-06 Thread qihua wu
I used the following command to install the numpy to enable the SSE3 numpy-1.5.1rc1-win32-superpack-python3.1.exe /arch sse3 Then how can I know whether numpy is running with SSE or not? I have a program to process the data from sql server using java to process 600M rows, it takes 7 hours to comp