Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Citi, Luca
I tried to implement what you suggest. The patch is in the ticket page. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Robert Kern
On Thu, Jul 9, 2009 at 15:20, Fons Adriaensen wrote: > On Thu, Jul 09, 2009 at 12:00:23PM -0500, Robert Kern wrote: > >> On Thu, Jul 9, 2009 at 11:44, Fons Adriaensen wrote: >> >> > There is a simple rule which says that if you use an object >> > pointer as a function argument you must INCREF it. T

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Fons Adriaensen
On Thu, Jul 09, 2009 at 12:00:23PM -0500, Robert Kern wrote: > On Thu, Jul 9, 2009 at 11:44, Fons Adriaensen wrote: > > > There is a simple rule which says that if you use an object > > pointer as a function argument you must INCREF it. This is > > just the logical consequence of using refcounted

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Pauli Virtanen
On 2009-07-09, Citi, Luca wrote: > Let me see if I understand correctly... > what you suggest is something like: > 1) adding an argument flag to construct_arrays > that enables/disables the feature > 2) adding the same argument flag to construct_loop which > is passed untouched to construct_ar

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Citi, Luca
Let me see if I understand correctly... what you suggest is something like: 1) adding an argument flag to construct_arrays that enables/disables the feature 2) adding the same argument flag to construct_loop which is passed untouched to construct_arrays 3) set the flag to "disable" in the const

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Robert Kern
On Thu, Jul 9, 2009 at 11:44, Fons Adriaensen wrote: > On Thu, Jul 09, 2009 at 04:30:11PM +0100, Citi, Luca wrote: > >> Also one could have an internal flag which says whether or not is safe >> to overwrite inputs with ref_count=1. >> Then import_array() sets this flag to "unsafe" (i.e. current beh

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Fons Adriaensen
On Thu, Jul 09, 2009 at 04:30:11PM +0100, Citi, Luca wrote: > Also one could have an internal flag which says whether or not is safe > to overwrite inputs with ref_count=1. > Then import_array() sets this flag to "unsafe" (i.e. current behaviour). > If the user of the numpy C-api is aware of how t

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Gael Varoquaux
On Thu, Jul 09, 2009 at 10:41:38AM -0500, Robert Kern wrote: > We could change ufunc_generic_call() (i.e. the C implementation of > ufunc.__call__) to use a new function like PyUFunc_GenericFunction > except with the refcount-1 semantics. This allows old C-level to > remain unchanged but let Python

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Robert Kern
On Thu, Jul 9, 2009 at 04:35, Pauli Virtanen wrote: > Thu, 09 Jul 2009 10:03:47 +0100, Citi, Luca kirjoitti: > [clip] >> Excuse me if I insist, PyArray_Conjugate is not the problem. If when >> using the numpy API, it is accepted something like: >> >>     obj1 = PyArray_CreateSomehowAnArray(); >

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Robert Kern
On Thu, Jul 9, 2009 at 10:30, Citi, Luca wrote: > Hello Gaël, > > I think it might be an option. > > Also one could have an internal flag which says whether or not is safe > to overwrite inputs with ref_count=1. > Then import_array() sets this flag to "unsafe" (i.e. current behaviour). > If the use

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Citi, Luca
Hello Gaël, I think it might be an option. Also one could have an internal flag which says whether or not is safe to overwrite inputs with ref_count=1. Then import_array() sets this flag to "unsafe" (i.e. current behaviour). If the user of the numpy C-api is aware of how the new feature works, he

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Gael Varoquaux
On Thu, Jul 09, 2009 at 09:35:06AM +, Pauli Virtanen wrote: > Thu, 09 Jul 2009 10:03:47 +0100, Citi, Luca kirjoitti: > [clip] > > Excuse me if I insist, PyArray_Conjugate is not the problem. If when > > using the numpy API, it is accepted something like: > > obj1 = PyArray_CreateSomehowAnA

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Pauli Virtanen
Thu, 09 Jul 2009 10:03:47 +0100, Citi, Luca kirjoitti: [clip] > Excuse me if I insist, PyArray_Conjugate is not the problem. If when > using the numpy API, it is accepted something like: > > obj1 = PyArray_CreateSomehowAnArray(); > obj2 = PyArray_DoSomethingWithArray(obj1,...); > o

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Citi, Luca
Hello Pauli, excuse me if I insist, PyArray_Conjugate is not the problem. If when using the numpy API, it is accepted something like: ob1 = PyArray_CreateSomehowAnArray(); obj2 = PyArray_DoSomethingWithArray(obj1,...); obj3 = PyArray_DoSomethingElseWithArray(obj1,...); Py_DECREF(obj1); then

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Pauli Virtanen
Thu, 09 Jul 2009 10:00:25 +0200, Matthieu Brucher kirjoitti: > 2009/7/9 Citi, Luca : >> Hello >> >> The problem is not PyArray_Conjugate itself. The problem is that >> whenever you call a function from the C side and one of the inputs has >> ref_count 1, it can be overwritten. This is not a proble

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-09 Thread Matthieu Brucher
2009/7/9 Citi, Luca : > Hello > > The problem is not PyArray_Conjugate itself. > The problem is that whenever you call a function from the C side > and one of the inputs has ref_count 1, it can be overwritten. > This is not a problem from the python side because if the > ufunc sees a ref_count=1 it

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
Hello The problem is not PyArray_Conjugate itself. The problem is that whenever you call a function from the C side and one of the inputs has ref_count 1, it can be overwritten. This is not a problem from the python side because if the ufunc sees a ref_count=1 it means that no python object is ref

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Robert Kern
On Wed, Jul 8, 2009 at 18:28, Pauli Virtanen wrote: > On 2009-07-08, Charles R Harris wrote: >> In that case I don't see a problem offhand. That said, I haven't looked at >> the code yet. > > I'm a bit worried about the problem that cropped up in the ticket > with the complex ufuncs. As Luca noted

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Pauli Virtanen
On 2009-07-08, Charles R Harris wrote: > In that case I don't see a problem offhand. That said, I haven't looked at > the code yet. I'm a bit worried about the problem that cropped up in the ticket with the complex ufuncs. As Luca noted in the ticket, obj3 = PyArray_Conjugate((PyAO *)obj1,

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 4:17 PM, Citi, Luca wrote: > > > On thing to keep in mind is that the inputs might be different views of > the > > same array so the elements might accessed in an unexpected order. > > Only inputs owning their own data and with refcount 1 (i.e. no other array > can be a vie

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
> On thing to keep in mind is that the inputs might be different views of the > same array so the elements might accessed in an unexpected order. Only inputs owning their own data and with refcount 1 (i.e. no other array can be a view of it) are re-used as outputs. __

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 4:10 PM, Citi, Luca wrote: > @Charles R Harris > > >> For example 'sqrt(a**2 + b**2)' can be performed... > > I think this particular function is already available as a ufunc. > > I am not sure it is implemented as ufunc. > But in any case it was just an example. > > Anothe

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Robert Kern
On Wed, Jul 8, 2009 at 17:10, Citi, Luca wrote: > @Charles R Harris > >>> For example 'sqrt(a**2 + b**2)' can be performed... >> I think this particular function is already available as a ufunc. > > I am not sure it is implemented as ufunc. hypot(a, b) -- Robert Kern "I have come to believe tha

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
@Charles R Harris >> For example 'sqrt(a**2 + b**2)' can be performed... > I think this particular function is already available as a ufunc. I am not sure it is implemented as ufunc. But in any case it was just an example. Another example is sin(2*pi*w+phi) that is currently implemented allocat

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 3:57 PM, Citi, Luca wrote: > Hi Stefan, > I am afraid I did not explain myself clear enough. > > Of course > c = a + b + d > leaves a, b, and d unchanged. > The only array that is overwritten is (a+b) which is a temporary > array that would be destroyed anyway. > Normally t

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
Hi Stefan, I am afraid I did not explain myself clear enough. Of course c = a + b + d leaves a, b, and d unchanged. The only array that is overwritten is (a+b) which is a temporary array that would be destroyed anyway. Normally the operation above is performed like this: 1) allocation of a tempora

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Charles R Harris
On Wed, Jul 8, 2009 at 11:34 AM, Citi, Luca wrote: > Hello guys, > I made a patch for numpy which allows performing > operations in-place to save memory allocations. > For example 'sqrt(a**2 + b**2)' can be performed I think this particular function is already available as a ufunc. What might b

Re: [Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Stéfan van der Walt
Hi Luca 2009/7/8 Citi, Luca : > Hello guys, > I made a patch for numpy which allows performing > operations in-place to save memory allocations. > For example 'sqrt(a**2 + b**2)' can be performed > allocating only two arrays instead of four. > You find the details in ticket 1153 of numpy-core. > I

[Numpy-discussion] performing operations in-place in numpy

2009-07-08 Thread Citi, Luca
Hello guys, I made a patch for numpy which allows performing operations in-place to save memory allocations. For example 'sqrt(a**2 + b**2)' can be performed allocating only two arrays instead of four. You find the details in ticket 1153 of numpy-core. I thought maybe you could be interested. I am