Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Matthew Brett
Hi, On Tue, Oct 8, 2013 at 12:44 PM, Matthew Brett wrote: > Hi, > > On Tue, Oct 8, 2013 at 4:38 AM, Ke Sun wrote: >> On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote: >>> Hi, >>> >>> On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun wrote: >>> > Dear all, >>> > >>> > I have written the follow

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Matthew Brett
Hi, On Tue, Oct 8, 2013 at 4:38 AM, Ke Sun wrote: > On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote: >> Hi, >> >> On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun wrote: >> > Dear all, >> > >> > I have written the following function to compute the square distances of a >> > large >> > matri

Re: [Numpy-discussion] Question about typenum

2013-10-08 Thread Valentin Haenel
* Richard Hattersley [2013-10-08]: > On 8 October 2013 19:56, Valentin Haenel wrote: > > > I ended up using: PyArray_TypeObjectFromType > > from cython so: > > > > np.dtype(cnp.PyArray_TypeObjectFromType(self.ndtype)).str > > > > Maybe i can avoid the np.dtype call, when using PyArray_Descr? > >

Re: [Numpy-discussion] Question about typenum

2013-10-08 Thread Richard Hattersley
On 8 October 2013 19:56, Valentin Haenel wrote: > I ended up using: PyArray_TypeObjectFromType > from cython so: > > np.dtype(cnp.PyArray_TypeObjectFromType(self.ndtype)).str > > Maybe i can avoid the np.dtype call, when using PyArray_Descr? > In short: yes. `PyArray_TypeObjectFromType` first u

Re: [Numpy-discussion] Bug in numpy.correlate documentation

2013-10-08 Thread Richard Hattersley
Hi Bernard, Looks like you're on to something - two other people have raised this discrepancy before: https://github.com/numpy/numpy/issues/2588. Unfortunately, when it comes to resolving the discrepancy one of the previous comments takes the opposite view. Namely, that the docstring is correct an

Re: [Numpy-discussion] Question about typenum

2013-10-08 Thread Valentin Haenel
Hi Richard, * Richard Hattersley [2013-10-08]: > On 8 October 2013 13:23, Valentin Haenel wrote: > > > Certain functions, like > > `PyArray_SimpleNewFromData` `PyArray_SimpleNew` take a typeenum > > Is there any way to go from typeenum to something that can be > > passed to the dtype constructo

Re: [Numpy-discussion] Question about typenum

2013-10-08 Thread Richard Hattersley
Hi Valentin, On 8 October 2013 13:23, Valentin Haenel wrote: > Certain functions, like > `PyArray_SimpleNewFromData` `PyArray_SimpleNew` take a typeenum > Is there any way to go from typeenum to something that can be > passed to the dtype constructor, like mapping 12 -> ' If you just want the c

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Jaime Fernández del Río
On Tue, Oct 8, 2013 at 4:38 AM, Ke Sun wrote: > On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote: > > Hi, > > > > On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun wrote: > > > Dear all, > > > > > > I have written the following function to compute the square distances > of a large > > > matrix

[Numpy-discussion] Python function for line intersection??

2013-10-08 Thread Happyman
Hello, I am having trouble with solving line intersection problem. I would solve it using standard functions which may already exist in Python library. Let me explain it below: I have sets of lines A(x,y) and B(x,y) and one map as a matrix form, also with its resolution dx, dy which can be cha

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Julian Taylor
On Tue, Oct 8, 2013 at 1:38 PM, Ke Sun wrote: > > > On a machine I had access to it took about 20 minutes. > How? I am using matrix multiplication (the same code as > http://stackoverflow.com/a/4856692) and it runs for around 18 hours. > > make sure you are using an optimized BLAS library. e.g. w

[Numpy-discussion] Question about typenum

2013-10-08 Thread Valentin Haenel
Hi, I have a quick question about typenum. Certain functions, like `PyArray_SimpleNewFromData` `PyArray_SimpleNew` take a typeenum argument. Is there any way to go from typeenum to something that can be passed to the dtype constructor, like mapping 12 -> 'http://mail.scipy.org/mailman/listinfo/nu

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Ke Sun
On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote: > Hi, > > On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun wrote: > > Dear all, > > > > I have written the following function to compute the square distances of a > > large > > matrix (each sample a row). It compute row by row and print the ov

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Julian Taylor
> Out of interest, how did you do this with matrix multiplication? http://stackoverflow.com/a/4856692 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Henry Gomersall
On 08/10/13 09:49, Matthew Brett wrote: > On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun wrote: >> >Dear all, >> > >> >I have written the following function to compute the square distances of a >> >large >> >matrix (each sample a row). It compute row by row and print the overall >> >progress. >> >The pr

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Henry Gomersall
On 08/10/13 09:06, Ke Sun wrote: > I give as input a 70,000x800 matrix. The output should be a 70,000x70,000 > matrix. The program runs really slow (16 hours for 1/3 progress). And it eats > 36G memory (fortunately I have enough). At this stage I'd be asking myself what I'm trying to achieve and w

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Julian Taylor
your computation is symmetric so you only need to compute the upper or lower triangle which will save both memory and time. On Tue, Oct 8, 2013 at 10:06 AM, Ke Sun wrote: > Dear all, > > I have written the following function to compute the square distances of a > large > matrix (each sample a r

Re: [Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Matthew Brett
Hi, On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun wrote: > Dear all, > > I have written the following function to compute the square distances of a > large > matrix (each sample a row). It compute row by row and print the overall > progress. > The progress output is important and I didn't use matrix m

[Numpy-discussion] Efficient square distance computation

2013-10-08 Thread Ke Sun
Dear all, I have written the following function to compute the square distances of a large matrix (each sample a row). It compute row by row and print the overall progress. The progress output is important and I didn't use matrix multiplication. I give as input a 70,000x800 matrix. The output sh