Re: [Numpy-discussion] Index Array Performance

2012-02-14 Thread Pauli Virtanen
15.02.2012 00:23, Marcel Oliver kirjoitti: [clip] > Thanks for all the replies. Playing a bit with timeit, it is clear > that it cannot just be the overhead of checking the type of the index > array, as the overhead grows very roughly propertional to the size of > the index array, but remains inde

Re: [Numpy-discussion] Index Array Performance

2012-02-14 Thread Wes McKinney
On Tue, Feb 14, 2012 at 4:03 AM, Francesc Alted wrote: > On Feb 14, 2012, at 1:50 AM, Wes McKinney wrote: > [clip] >> But: >> >> In [40]: timeit hist[i, j] >> 1 loops, best of 3: 32 us per loop >> >> So that's roughly 7-8x slower than a simple Cython method, so I >> sincerely hope it could be

Re: [Numpy-discussion] Index Array Performance

2012-02-14 Thread Marcel Oliver
Francesc Alted wrote: > On Feb 14, 2012, at 1:50 AM, Wes McKinney wrote: > [clip] > > But: > > > > In [40]: timeit hist[i, j] > > 1 loops, best of 3: 32 us per loop > > > > So that's roughly 7-8x slower than a simple Cython method, so I > > sincerely hope it could be brought down to

Re: [Numpy-discussion] Index Array Performance

2012-02-14 Thread Robert Kern
On Mon, Feb 13, 2012 at 23:23, Marcel Oliver wrote: > Hi, > > I have a short piece of code where the use of an index array "feels > right", but incurs a severe performance penalty: It's about an order > of magnitude slower than all other operations with arrays of that > size. > > It comes up in a

Re: [Numpy-discussion] Index Array Performance

2012-02-14 Thread Francesc Alted
On Feb 14, 2012, at 1:50 AM, Wes McKinney wrote: [clip] > But: > > In [40]: timeit hist[i, j] > 1 loops, best of 3: 32 us per loop > > So that's roughly 7-8x slower than a simple Cython method, so I > sincerely hope it could be brought down to the sub 10 microsecond > level with a little bit

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
On Mon, Feb 13, 2012 at 7:48 PM, Wes McKinney wrote: > On Mon, Feb 13, 2012 at 7:46 PM, Wes McKinney wrote: >> On Mon, Feb 13, 2012 at 7:30 PM, Nathaniel Smith wrote: >>> How would you fix it? I shouldn't speculate without profiling, but I'll be >>> naughty. Presumably the problem is that python

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
On Mon, Feb 13, 2012 at 7:46 PM, Wes McKinney wrote: > On Mon, Feb 13, 2012 at 7:30 PM, Nathaniel Smith wrote: >> How would you fix it? I shouldn't speculate without profiling, but I'll be >> naughty. Presumably the problem is that python turns that into something >> like >> >> hist[i,j] = hist[i

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
On Mon, Feb 13, 2012 at 7:30 PM, Nathaniel Smith wrote: > How would you fix it? I shouldn't speculate without profiling, but I'll be > naughty. Presumably the problem is that python turns that into something > like > > hist[i,j] = hist[i,j] + 1 > > which means there's no way for numpy to avoid cre

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Nathaniel Smith
How would you fix it? I shouldn't speculate without profiling, but I'll be naughty. Presumably the problem is that python turns that into something like hist[i,j] = hist[i,j] + 1 which means there's no way for numpy to avoid creating a temporary array. So maybe this could be fixed by adding a fus

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
On Mon, Feb 13, 2012 at 6:23 PM, Marcel Oliver wrote: > Hi, > > I have a short piece of code where the use of an index array "feels > right", but incurs a severe performance penalty: It's about an order > of magnitude slower than all other operations with arrays of that > size. > > It comes up in

[Numpy-discussion] Index Array Performance

2012-02-13 Thread Marcel Oliver
Hi, I have a short piece of code where the use of an index array "feels right", but incurs a severe performance penalty: It's about an order of magnitude slower than all other operations with arrays of that size. It comes up in a piece of code which is doing a large number of "on the fly" histogr