[Numpy-discussion] take from structured array is faster than boolean indexing, but reshapes columns to 2D

2010-12-21 Thread Christopher Mutel
Dear all- Structured arrays are great, but I am having problems filtering them efficiently. Reading through the mailing list, it seems like boolean arrays are the recommended approach to filtering arrays for arbitrary conditions, but my testing shows that a combination of take and where can be muc

Re: [Numpy-discussion] NEP for faster ufuncs

2010-12-21 Thread Charles R Harris
On Tue, Dec 21, 2010 at 5:53 PM, Mark Wiebe wrote: > Hello NumPy-ers, > > After some performance analysis, I've designed and implemented a new > iterator designed to speed up ufuncs and allow for easier multi-dimensional > iteration. The new code is fairly large, but works quite well already. I

Re: [Numpy-discussion] NEP for faster ufuncs

2010-12-21 Thread Mark Wiebe
On Tue, Dec 21, 2010 at 6:06 PM, David wrote: > > This looks pretty cool. I hope to be able to take a look at it during > the christmas holidays. > Thanks! > > I cannot comment in details yet, but it seems to address several issues > I encountered myself while implementing the neighborhood iter

Re: [Numpy-discussion] NEP for faster ufuncs

2010-12-21 Thread Mark Wiebe
That's a good suggestion - added. Unfortunately, it looks like the github rst converter doesn't make a table of contents with working links. Cheers, Mark On Tue, Dec 21, 2010 at 6:00 PM, John Salvatier wrote: > I applaud you on your vision. I only have one small suggestion: I suggest > you put

Re: [Numpy-discussion] NEP for faster ufuncs

2010-12-21 Thread David
Hi Mark, On 12/22/2010 09:53 AM, Mark Wiebe wrote: > Hello NumPy-ers, > > After some performance analysis, I've designed and implemented a new > iterator designed to speed up ufuncs and allow for easier > multi-dimensional iteration. The new code is fairly large, but works > quite well already.

Re: [Numpy-discussion] NEP for faster ufuncs

2010-12-21 Thread John Salvatier
I applaud you on your vision. I only have one small suggestion: I suggest you put a table of contents at the beginning of your NEP so people may skip to the part that most interests them. On Tue, Dec 21, 2010 at 4:59 PM, John Salvatier wrote: > That is an amazing christmas present. > > On Tue, De

Re: [Numpy-discussion] Giving numpy the ability to multi-iterate excluding an axis

2010-12-21 Thread Mark Wiebe
On Mon, Dec 20, 2010 at 1:42 PM, John Salvatier wrote: > A while ago, I asked a whether it was possible to multi-iterate over > several ndarrays but exclude a certain axis( > http://www.mail-archive.com/numpy-discussion@scipy.org/msg29204.html), > sort of a combination of PyArray_IterAllButAxis an

Re: [Numpy-discussion] NEP for faster ufuncs

2010-12-21 Thread John Salvatier
That is an amazing christmas present. On Tue, Dec 21, 2010 at 4:53 PM, Mark Wiebe wrote: > Hello NumPy-ers, > > After some performance analysis, I've designed and implemented a new > iterator designed to speed up ufuncs and allow for easier multi-dimensional > iteration. The new code is fairly

[Numpy-discussion] NEP for faster ufuncs

2010-12-21 Thread Mark Wiebe
Hello NumPy-ers, After some performance analysis, I've designed and implemented a new iterator designed to speed up ufuncs and allow for easier multi-dimensional iteration. The new code is fairly large, but works quite well already. If some people could read the NEP and give some feedback, that

Re: [Numpy-discussion] sample without replacement

2010-12-21 Thread Robert Kern
On Mon, Dec 20, 2010 at 10:28, Alan G Isaac wrote: > I want to sample *without* replacement from a vector > (as with Python's random.sample).  I don't see a direct > replacement for this, and I don't want to carry two > PRNG's around.  Is the best way something like  this? > >        permutation(m

Re: [Numpy-discussion] Reversing an array in-place

2010-12-21 Thread Sturla Molden
Chuck wrote: > The reversed matrix is a view, no copyihg is done. It is even faster than > an inplace reversal. This is why I love NumPy. In C, Fortran or Matlab most programmers would probably form the reversed array. In NumPy we just change some metainformation (data pointer and strides) behin

Re: [Numpy-discussion] sample without replacement

2010-12-21 Thread Sturla Molden
We often need to generate more than one such sample from an array, e.g. for permutation tests. If we shuffle an array x of size N and use x[:M] as a random sample "without replacement", we just need to put them back randomly to get the next sample (cf. Fisher-Yates shuffle). That way we get O(M) a

[Numpy-discussion] bincount and generators

2010-12-21 Thread Alan G Isaac
bincount does not currently allow a generator as an argument. I'm wondering if it is considered too costly to extend it to allow this. (Motivation: I'm counting based on an attribute of a large number of objects, and I don't need a list of the data.) Thanks, Alan Isaac ___

[Numpy-discussion] bincount question

2010-12-21 Thread Alan G Isaac
:: >>> np.bincount([]) Traceback (most recent call last): File "", line 1, in ValueError: The first argument cannot be empty. Why not? (I.e., why isn't an empty array the right answer?) Thanks, Alan Isaac ___ NumPy-Discussion ma

Re: [Numpy-discussion] sample without replacement

2010-12-21 Thread Anne Archibald
I know this question came up on the mailing list some time ago (19/09/2008), and the conclusion was that yes, you can do it more or less efficiently in pure python; the trick is to use two different methods. If your sample is more than, say, a quarter the size of the set you're drawing from, you pe

Re: [Numpy-discussion] sample without replacement

2010-12-21 Thread Alan G Isaac
On 12/20/2010 10:49 PM, josef.p...@gmail.com wrote: > What's the difference between a numpy Random and a python > random.Random instance of separate states of the random number > generators? Sorry, I don't understand the question. The difference for my use is that a np.RandomState instance provi