Re: [Numpy-discussion] argwhere does not accept py list

2009-07-08 Thread Scott Sinclair
> 2009/7/8 Pierre GM : > > On Jul 8, 2009, at 3:18 AM, Scott Sinclair wrote: > >>> 2009/7/8 Robert Kern : >>> 2009/7/4 Stéfan van der Walt : Thanks, Scott.  This should now be fixed in SVN. >>> >>> You should probably change that to asanyarray() before the masked >>> array crowd gets upset. :-

[Numpy-discussion] speeding up an array operation

2009-07-08 Thread Mag Gam
Hey All I am reading thru a file and trying to store the values into another array, but instead of storing the values 1 by 1, I would like to store them in bulk sets for optimization purposes. Here is what I have, which does it 1x1: z={} #dictionary r=csv.reader(file) for i,row in enumerate(r):

Re: [Numpy-discussion] Bug in the F distribution?

2009-07-08 Thread alan
> >>On Fri, Jul 3, 2009 at 10:21 PM, Alan Jackson wrote: >>> >> >>I don't see any problem here. If you can replicate your results, we >>would need more information about the versions. >> >>Josef >> >>''' > np.version.version >>'1.3.0' > scipy.version.version >>'0.8.0.dev5789' >>''' >> > >In

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
Hello The problem is not PyArray_Conjugate itself. The problem is that whenever you call a function from the C side and one of the inputs has ref_count 1, it can be overwritten. This is not a problem from the python side because if the ufunc sees a ref_count=1 it means that no python object is ref

Re: [Numpy-discussion] Help with np.where and datetime functions

2009-07-08 Thread Robert Kern
On Wed, Jul 8, 2009 at 19:11, John [H2O] wrote: > > Also, could someone please explain why: > > Tsub = T[ (T[:,0]>t1) & (T[:,0] > Works, but: > > Tsub = T[ (t1 > Does not??? I'm not positive, but I think it boils down to this: Python tries to look up the method from the first operand before the se

Re: [Numpy-discussion] Help with np.where and datetime functions

2009-07-08 Thread John [H2O]
Also, could someone please explain why: Tsub = T[ (T[:,0]>t1) & (T[:,0]http://www.nabble.com/Help-with-np.where-and-datetime-functions-tp24389447p24401687.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ NumPy-Discussion mail

Re: [Numpy-discussion] Help with np.where and datetime functions

2009-07-08 Thread John [H2O]
nhmc wrote: > > > > Also, if you don't need the indices, you can just use the conditional > expression as a boolean mask: > condition = (t1 < Y[:,0]) & (Y[:,0] < t2) Y[:,0][condition] > > Neil > 'condition' is not an index array? Wouldn't it just be the indices as well? Would i

Re: [Numpy-discussion] Help with np.where and datetime functions

2009-07-08 Thread John [H2O]
Pierre GM-2 wrote: > > > > Would you like to give the scikits.timeseries package a try ? It's > available at pytseries.sourceforge.net. > Calculatng the hourly average should be straightforward. > I would, in fact I have been investigating it, but I didn't have numpy1.3 up and running unti

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Robert Kern
On Wed, Jul 8, 2009 at 18:28, Pauli Virtanen wrote: > On 2009-07-08, Charles R Harris wrote: >> In that case I don't see a problem offhand. That said, I haven't looked at >> the code yet. > > I'm a bit worried about the problem that cropped up in the ticket > with the complex ufuncs. As Luca noted

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Pauli Virtanen
On 2009-07-08, Charles R Harris wrote: > In that case I don't see a problem offhand. That said, I haven't looked at > the code yet. I'm a bit worried about the problem that cropped up in the ticket with the complex ufuncs. As Luca noted in the ticket, obj3 = PyArray_Conjugate((PyAO *)obj1,

Re: [Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 5:02 PM, Pauli Virtanen > wrote: > On 2009-07-08, Stéfan van der Walt wrote: > > I know very little about cache optimality, so excuse the triviality of > > this question: Is it possible to design this loop optimally (taking > > into account certain build-time measurable pa

[Numpy-discussion] ndarray and the buffer protocol

2009-07-08 Thread Fons Adriaensen
Hello all, (resending for the Nth time, as the previous attempts didn't make it to the list) I'm new to this list (and numpy is mostly new to me :-). Using python 2.6 and numpy 1.3. My plan is to write some C extensions that will perform rather specialised processing on multichannel digital au

Re: [Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread David Cournapeau
On Thu, Jul 9, 2009 at 8:02 AM, Pauli Virtanen wrote: > > I don't think we want to go the ATNumPy route, or even have > tunable parameters chosen at build or compile time. Detecting things like cache size at compile time should not be too difficult, at least for common platforms. Even detecting i

Re: [Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread Robert Kern
On Wed, Jul 8, 2009 at 18:02, Pauli Virtanen wrote: > On 2009-07-08, Stéfan van der Walt wrote: >> I know very little about cache optimality, so excuse the triviality of >> this question: Is it possible to design this loop optimally (taking >> into account certain build-time measurable parameters)

Re: [Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread Pauli Virtanen
On 2009-07-08, Stéfan van der Walt wrote: > I know very little about cache optimality, so excuse the triviality of > this question: Is it possible to design this loop optimally (taking > into account certain build-time measurable parameters), or is it the > kind of thing that can only be discovere

Re: [Numpy-discussion] Problem with ML subscriptions?

2009-07-08 Thread Robert Kern
On Wed, Jul 8, 2009 at 17:53, Fons Adriaensen wrote: > On Wed, Jul 08, 2009 at 11:01:55PM +0100, Peter wrote: >> Anyway - fingers crossed the list is working for you >> now... > > I'm not convinced... Will try posting again. And here you are. -- Robert Kern "I have come to believe that the who

Re: [Numpy-discussion] Problem with ML subscriptions?

2009-07-08 Thread Fons Adriaensen
On Wed, Jul 08, 2009 at 11:01:55PM +0100, Peter wrote: > On Wed, Jul 8, 2009 at 10:00 PM, Fons Adriaensen wrote: > > > > On Sun, Jul 05, 2009 at 12:22:37AM +0200, Stéfan van der Walt wrote: > > > >> 2009/7/5 Pauli Virtanen : > >> > 2009-07-04 22:52 +0200, Fons Adriaensen wrote: > >> > [clip] > >>

Re: [Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread Pauli Virtanen
On 2009-07-08, Charles R Harris wrote: [clip] > How do the benchmarks compare with just making contiguous copies? Which is > blocking of sort, I suppose. I think that's slower than just walking over the discontiguous array: 1) The new code: (on the Athlon machine) $ ./bench-red

Re: [Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread Stéfan van der Walt
Hi Pauli 2009/7/9 Pauli Virtanen : > Unfortunately, improving the performance using the above scheme > comes at the cost of some slightly murky heuristics.  I didn't > manage to come up with an optimal decision rule, so they are > partly empirical. There is one parameter tuning the cross-over > be

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 4:17 PM, Citi, Luca wrote: > > > On thing to keep in mind is that the inputs might be different views of > the > > same array so the elements might accessed in an unexpected order. > > Only inputs owning their own data and with refcount 1 (i.e. no other array > can be a vie

Re: [Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 4:16 PM, Pauli Virtanen > wrote: > Hi, > > Ticket #1143 points out that Numpy's reduction operations are not > always cache friendly. I worked a bit on tuning them. > > > Just to tickle some interest, a "pathological" case before optimization: > >In [1]: import numpy as

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
> On thing to keep in mind is that the inputs might be different views of the > same array so the elements might accessed in an unexpected order. Only inputs owning their own data and with refcount 1 (i.e. no other array can be a view of it) are re-used as outputs. __

[Numpy-discussion] Optimizing reduction loops (sum(), prod(), et al.)

2009-07-08 Thread Pauli Virtanen
Hi, Ticket #1143 points out that Numpy's reduction operations are not always cache friendly. I worked a bit on tuning them. Just to tickle some interest, a "pathological" case before optimization: In [1]: import numpy as np In [2]: x = np.zeros((8, 256)) In [3]: %timeit x.sum(ax

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 4:10 PM, Citi, Luca wrote: > @Charles R Harris > > >> For example 'sqrt(a**2 + b**2)' can be performed... > > I think this particular function is already available as a ufunc. > > I am not sure it is implemented as ufunc. > But in any case it was just an example. > > Anothe

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Robert Kern
On Wed, Jul 8, 2009 at 17:10, Citi, Luca wrote: > @Charles R Harris > >>> For example 'sqrt(a**2 + b**2)' can be performed... >> I think this particular function is already available as a ufunc. > > I am not sure it is implemented as ufunc. hypot(a, b) -- Robert Kern "I have come to believe tha

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
@Charles R Harris >> For example 'sqrt(a**2 + b**2)' can be performed... > I think this particular function is already available as a ufunc. I am not sure it is implemented as ufunc. But in any case it was just an example. Another example is sin(2*pi*w+phi) that is currently implemented allocat

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 3:57 PM, Citi, Luca wrote: > Hi Stefan, > I am afraid I did not explain myself clear enough. > > Of course > c = a + b + d > leaves a, b, and d unchanged. > The only array that is overwritten is (a+b) which is a temporary > array that would be destroyed anyway. > Normally t

Re: [Numpy-discussion] Problem with ML subscriptions?

2009-07-08 Thread Peter
On Wed, Jul 8, 2009 at 10:00 PM, Fons Adriaensen wrote: > > On Sun, Jul 05, 2009 at 12:22:37AM +0200, Stéfan van der Walt wrote: > >> 2009/7/5 Pauli Virtanen : >> > 2009-07-04 22:52 +0200, Fons Adriaensen wrote: >> > [clip] >> >> I subscribed to numpy-discussion almost two days ago. >> >> I do rec

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
Hi Stefan, I am afraid I did not explain myself clear enough. Of course c = a + b + d leaves a, b, and d unchanged. The only array that is overwritten is (a+b) which is a temporary array that would be destroyed anyway. Normally the operation above is performed like this: 1) allocation of a tempora

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 11:34 AM, Citi, Luca wrote: > Hello guys, > I made a patch for numpy which allows performing > operations in-place to save memory allocations. > For example 'sqrt(a**2 + b**2)' can be performed I think this particular function is already available as a ufunc. What might b

Re: [Numpy-discussion] Problem with ML subscriptions?

2009-07-08 Thread Fons Adriaensen
On Sun, Jul 05, 2009 at 12:22:37AM +0200, Stéfan van der Walt wrote: > 2009/7/5 Pauli Virtanen : > > 2009-07-04 22:52 +0200, Fons Adriaensen wrote: > > [clip] > >> I subscribed to numpy-discussion almost two days ago. > >> I do receive messages from the list, but the ones I post > >> don't appear

Re: [Numpy-discussion] Mathematica vs Matlab vs Python

2009-07-08 Thread Stéfan van der Walt
2009/7/8 Gael Varoquaux : > On Wed, Jul 08, 2009 at 12:48:17PM -0700, Matthew Brett wrote: >> > Yup. It's not even very idiomatic Python. readlines() is probably a >> > bad idea unless your file is trivial length, and even ignoring >> > numpy.loadtxt(), all of this could be considerably simplified

Re: [Numpy-discussion] argwhere does not accept py list

2009-07-08 Thread Stéfan van der Walt
2009/7/8 Robert Kern : > 2009/7/4 Stéfan van der Walt : >> Thanks, Scott.  This should now be fixed in SVN. > > You should probably change that to asanyarray() before the masked > array crowd gets upset. :-) Thanks, fixed! Stéfan ___ Numpy-discussion ma

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Stéfan van der Walt
Hi Luca 2009/7/8 Citi, Luca : > Hello guys, > I made a patch for numpy which allows performing > operations in-place to save memory allocations. > For example 'sqrt(a**2 + b**2)' can be performed > allocating only two arrays instead of four. > You find the details in ticket 1153 of numpy-core. > I

Re: [Numpy-discussion] Mathematica vs Matlab vs Python

2009-07-08 Thread Gael Varoquaux
On Wed, Jul 08, 2009 at 12:48:17PM -0700, Matthew Brett wrote: > > Yup. It's not even very idiomatic Python. readlines() is probably a > > bad idea unless your file is trivial length, and even ignoring > > numpy.loadtxt(), all of this could be considerably simplified with the > > built-in csv modul

Re: [Numpy-discussion] Mathematica vs Matlab vs Python

2009-07-08 Thread Matthew Brett
Hi, > Yup. It's not even very idiomatic Python. readlines() is probably a > bad idea unless your file is trivial length, and even ignoring > numpy.loadtxt(), all of this could be considerably simplified with the > built-in csv module. or a 1-liner with scipy.io.loadmat ... Best, Matthew ___

Re: [Numpy-discussion] Mathematica vs Matlab vs Python

2009-07-08 Thread David Warde-Farley
On 8-Jul-09, at 4:19 AM, Pauli Virtanen wrote: > Tue, 07 Jul 2009 21:30:06 -0500, alan kirjoitti: >> Mathematica vs Matlab vs Python >> >> http://www.larssono.com/musings/matmatpy/index.html > > The Python code there is not very idiomatic Numpy code. It's written > for > Numeric, and fancy index

Re: [Numpy-discussion] Help with np.where and datetime functions

2009-07-08 Thread Pierre GM
On Jul 8, 2009, at 7:03 AM, John [H2O] wrote: > > Hello, > > I have several issues which require me to iterate through a fairly > large > array (30+ records). > > The first case is calculating and hourly average from non-regularly > sampled > data. Would you like to give the scikits.time

Re: [Numpy-discussion] argwhere does not accept py list

2009-07-08 Thread Pierre GM
On Jul 8, 2009, at 3:18 AM, Scott Sinclair wrote: >> 2009/7/8 Robert Kern : >> 2009/7/4 Stéfan van der Walt : >>> Thanks, Scott. This should now be fixed in SVN. >> >> You should probably change that to asanyarray() before the masked >> array crowd gets upset. :-) > > I hadn't thought about that

Re: [Numpy-discussion] Creating PyArrayObject in C++

2009-07-08 Thread Kenny Abernathy
Thanks, you two. That helps alot. The PyCObject_FromVoidPtr() trick is good to know--I still have to have my class because it holds other data, but this definitely points me in the right direction. On Tue, Jul 7, 2009 at 9:55 PM, Lisandro Dalcin wrote: > 2009/7/7 Stéfan van der Walt : > > Hi K

[Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
Hello guys, I made a patch for numpy which allows performing operations in-place to save memory allocations. For example 'sqrt(a**2 + b**2)' can be performed allocating only two arrays instead of four. You find the details in ticket 1153 of numpy-core. I thought maybe you could be interested. I am

Re: [Numpy-discussion] Help with np.where and datetime functions

2009-07-08 Thread Neil Crighton
John [H2O] gmail.com> writes: > What I am trying to do (obviously?) is find all the values of X that fall > within a time range. > > Specifically, one point I do not understand is why the following two methods > fail: > > --> 196 ind = np.where( (t1 < Y[:,0] < t2) ) #same result > with/

Re: [Numpy-discussion] roots and high-order polynomial

2009-07-08 Thread Fabrice Silva
Le lundi 06 juillet 2009 à 17:57 +0200, Fabrice Silva a écrit : > Le lundi 06 juillet 2009 à 17:13 +0200, Nils Wagner a écrit : > > IIRC, the coefficients of your polynomial are complex. > > So, you cannot guarantee that the roots are complex > > conjugate pairs. > > Correct! If the construction

Re: [Numpy-discussion] RFC: add a install_clib numpy.distutils command to install pure C libraries

2009-07-08 Thread David Cournapeau
On Fri, Jun 12, 2009 at 7:46 PM, David Cournapeau wrote: > Hi, > >    I have finally spent some time so that we can install pure C > libraries using numpy.distutils. With this, one could imagine having a C > library for fft, special functions in numpy or scipy, so that the > library could be reused

[Numpy-discussion] Help with np.where and datetime functions

2009-07-08 Thread John [H2O]
Hello, I have several issues which require me to iterate through a fairly large array (30+ records). The first case is calculating and hourly average from non-regularly sampled data. The second is screening one array, based on data in the second array. The functions are defined below, but in

Re: [Numpy-discussion] Question for David re npy_funcs

2009-07-08 Thread David Cournapeau
On Wed, Jul 8, 2009 at 5:37 AM, Charles R Harris wrote: > David, > > Should any standard c functions used in loops.c.src be the npy_* version? > I've been using fabs, but I'm wondering if that should be npy_fabs. Yes. Although fabs is available on any platform in theory, we should standardize on u

Re: [Numpy-discussion] Mathematica vs Matlab vs Python

2009-07-08 Thread Pauli Virtanen
Tue, 07 Jul 2009 21:30:06 -0500, alan kirjoitti: > Mathematica vs Matlab vs Python > > http://www.larssono.com/musings/matmatpy/index.html The Python code there is not very idiomatic Numpy code. It's written for Numeric, and fancy indexing etc. are not used. Seems like the author also left it a

Re: [Numpy-discussion] argwhere does not accept py list

2009-07-08 Thread Scott Sinclair
> 2009/7/8 Robert Kern : > 2009/7/4 Stéfan van der Walt : >> Thanks, Scott.  This should now be fixed in SVN. > > You should probably change that to asanyarray() before the masked > array crowd gets upset. :-) I hadn't thought about that, but I'm don't think it matters in this case. MaskedArray.no