Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-11 Thread Graeme B. Bell
>From my previous mail: >> this has the same performance as your code: >> a = empty([3] list(A.shape) For anyone that is interested. I ran a benchmark on the code after Julian kindly provided me with a correction to the listing he posted. >> a = empty([3] + list(A.shape)) >> a[0] = A>5; a[1] =

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Julian Taylor
hi, its not np.any that is slow in this case its np.array([A, B, C]) np.dstack([A, B, C]) is better but writing it like this has the same performance as your code: a = empty([3] list(A.shape) a[0] = A>5; a[1] = B<2; a[2] = A>10; np.any(a, 0) I'll check if creating an array from a sequence can be

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Chris Barker - NOAA Federal
This is good stuff, but I can't help thinking that if I needed to do an any/all test on a number of arrays with common and/or combos -- I'd probably write a Cython function to do it. It could be a bit tricky to make it really general, but not bad for a couple specific dtypes / use cases. -just a

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Graeme B. Bell
Hi Julian, Thanks for the post. It's great to hear that the main numpy function is improving in 1.8, though I think there is still plenty of value here for performance junkies :-) I don't have 1.8beta installed (and I can't conveniently install it on my machines just now). If you have ti

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Graeme B. Bell
Hi Robert, Thanks for proposing an alternative implementation approach. However, did you test your proposal before you made the assertion about its behaviour? >reduce(np.logical_or, inputs, False) >reduce(np.logical_and, inputs, True) This code consistently benchmarks 20% slower than the met

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Julian Taylor
On 04.09.2013 12:05, Graeme B. Bell wrote: > In my current GIS raster work I often have a situation where I generate code > something like this: > > np.any([A>4, A==2, B==5, ...]) > > However, np.any() is quite slow. > > It's possible to use np.logical_or to solve the problem, but the

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Robert Kern
On Wed, Sep 4, 2013 at 1:14 PM, Graeme B. Bell wrote: > > Sorry, I should have been more clear. > > As shown in the benchmark/example, the method is replacing the behaviour of > >np.any(inputs, 0) > > not the behaviour of > >np.any(inputs) reduce(np.logical_or, inputs, False) reduce(np.lo

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Phil Elson
For the record, I started a discussion about 6 months ago about a "find_first" type function which avoided running the logic over the whole array (using lambdas instead). This spilled into a discussion about implementing a short-cutted "any" or "all" function: http://numpy-discussion.10968.n7.nabbl

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Graeme B. Bell
Sorry, I should have been more clear. As shown in the benchmark/example, the method is replacing the behaviour of np.any(inputs, 0) not the behaviour of np.any(inputs) Here, where I'm making decisions based on overlaying layers of raster data in the same shape, I don't want to map the

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Robert Kern
On Wed, Sep 4, 2013 at 11:05 AM, Graeme B. Bell wrote: > > In my current GIS raster work I often have a situation where I generate code something like this: > > np.any([A>4, A==2, B==5, ...]) > > However, np.any() is quite slow. > > It's possible to use np.logical_or to solve the problem,

[Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Graeme B. Bell
In my current GIS raster work I often have a situation where I generate code something like this: np.any([A>4, A==2, B==5, ...]) However, np.any() is quite slow. It's possible to use np.logical_or to solve the problem, but then you get nested logical_or's, since logical_or combines o