Re: [Cython] Bindings performance issue
2011/6/2 Robert Bradshaw :
> On Wed, Jun 1, 2011 at 7:26 AM, Vitja Makarov wrote:
>> 2011/6/1 mark florisson :
>>> On 31 May 2011 20:25, Vitja Makarov wrote:
Hi!
Is bindings performance issue valuable?
$ cat bindbench.pyx
def wo_bindings():
pass
def outer():
def inner():
pass
return inner
with_bindings = outer()
$ python
>>> import timeit
>>> timeit.repeat('with_bindings()', setup='from bindbench import
>>> wo_bindings, with_bindings', repeat=1, number=1)
[6.169871807098389]
>>> timeit.repeat('wo_bindings()', setup='from bindbench import
>>> wo_bindings, with_bindings', repeat=1, number=1)
[4.609416961669922]
PyCBindings makes it 1.3 (difference is about 15ns on my laptop) times
slower for CPython interpreter execution.
As CPython has some optimizations for CFunctions and PyCFunctions.
Does it make sense for us? Or we can easily switch to bindings?
--
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>> I think switching should be fine, if you'd desperately need the speed
>>> you'd be calling c(p)def functions from Cython. In fact, when the
>>> fused cfunction will be ready it will be even slightly slower, as it
>>> overrides the tp_call. But perhaps that should really be made into a
>>> subclass...
>
> Yes, this should be a subclass, slowing all calls down is a bad idea.
>
>>> Anyway, would you use these for Python classes and module-level def
>>> functions (and closures) only, or would you also use them in extension
>>> classes? Because I found at least a bit of problem there, as extension
>>> class methods get 'self' passed as the first argument to the C
>>> function (as PyCFunctionObject.m_self), whereas methods in Python
>>> classes get 'self' passed in the args tuple (and the m_self is
>>> unused).
>>>
>>
>> Recently I've found a problem with static methods (__new__ for
>> example) of inner classes (classes created inside a function, for
>> instance)
>> I think binding version should be used for all regular def functions
>> (defs, staticmethods, classmethods and so on).
>
> +1 to moving that way. (Well, static and class methods bind
> differently, right?)
>
>> I also think that it's better to rename binding_PyCFunction_Type to
>> something like CyFunctionType and make it more like PyFunction.
>
> +1
>
> Is there any advantage to the method descriptors (other than that
> they're easier for a human to write?)
Initially bindings was written to support bound class methods (am I right?)
So when we use it for regular functions 'binding' in the name doesn't
reflect its purpose.
One the other hand it's much more easy to write manually.
About staticmethods: I think that CyFunction type should handle it as well
because staticmethods can have custom attributes and act just like
normal def one.
The difference is insde descr_get it should return bound method for
normal and self for staticmethods.
Here I've tried to support staticmethods inside bindings type:
https://github.com/vitek/cython/commit/c0725ab340a8173d8e6724c62be3a135df58980e
>
> I think that small speed regression for better compatibility is OK if
> we add a directive to not create binding functions. (It'd be nice if
> we could work around it, but that's hard as CPython has special
> hooks...) The bigger issue is that the binding behavior is backwards
> incompatible, and perhaps in a subtle way. Perhaps we need to have a
> phase where all currently non-binding functions that will become
> binding functions will raise an error (or at least a warning) to wean
> people off of the old behavior before making a switch.
>
Is the difference so significant?
--
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On Thu, Jun 2, 2011 at 1:30 AM, Vitja Makarov wrote:
> 2011/6/2 Robert Bradshaw :
>> On Wed, Jun 1, 2011 at 7:26 AM, Vitja Makarov
>> wrote:
>>> 2011/6/1 mark florisson :
On 31 May 2011 20:25, Vitja Makarov wrote:
> Hi!
>
> Is bindings performance issue valuable?
>
> $ cat bindbench.pyx
> def wo_bindings():
> pass
>
> def outer():
> def inner():
> pass
> return inner
> with_bindings = outer()
>
> $ python
import timeit
timeit.repeat('with_bindings()', setup='from bindbench import
wo_bindings, with_bindings', repeat=1, number=1)
> [6.169871807098389]
timeit.repeat('wo_bindings()', setup='from bindbench import
wo_bindings, with_bindings', repeat=1, number=1)
> [4.609416961669922]
>
> PyCBindings makes it 1.3 (difference is about 15ns on my laptop) times
> slower for CPython interpreter execution.
> As CPython has some optimizations for CFunctions and PyCFunctions.
>
> Does it make sense for us? Or we can easily switch to bindings?
>
> --
> vitja.
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>
I think switching should be fine, if you'd desperately need the speed
you'd be calling c(p)def functions from Cython. In fact, when the
fused cfunction will be ready it will be even slightly slower, as it
overrides the tp_call. But perhaps that should really be made into a
subclass...
>>
>> Yes, this should be a subclass, slowing all calls down is a bad idea.
>>
Anyway, would you use these for Python classes and module-level def
functions (and closures) only, or would you also use them in extension
classes? Because I found at least a bit of problem there, as extension
class methods get 'self' passed as the first argument to the C
function (as PyCFunctionObject.m_self), whereas methods in Python
classes get 'self' passed in the args tuple (and the m_self is
unused).
>>>
>>> Recently I've found a problem with static methods (__new__ for
>>> example) of inner classes (classes created inside a function, for
>>> instance)
>>> I think binding version should be used for all regular def functions
>>> (defs, staticmethods, classmethods and so on).
>>
>> +1 to moving that way. (Well, static and class methods bind
>> differently, right?)
>>
>>> I also think that it's better to rename binding_PyCFunction_Type to
>>> something like CyFunctionType and make it more like PyFunction.
>>
>> +1
>>
>> Is there any advantage to the method descriptors (other than that
>> they're easier for a human to write?)
>
> Initially bindings was written to support bound class methods (am I right?)
> So when we use it for regular functions 'binding' in the name doesn't
> reflect its purpose.
There's three kinds of functions we create: PyCFunctions, binding
PyCFunctions, and extension class methods (via descriptors). I was
asking about the latter.
As for the first two, it reflects the difference in behavior. If I
take the function and assign it to a class, it will bind as a method
when I go to look it up. This is arguably the biggest difference
between built-in functions and Python functions.
> One the other hand it's much more easy to write manually.
>
> About staticmethods: I think that CyFunction type should handle it as well
> because staticmethods can have custom attributes and act just like
> normal def one.
>
> The difference is insde descr_get it should return bound method for
> normal and self for staticmethods.
Yep. Does that merit a new class, or a flag that's checked on every
function invocation (false 99.9% of the time)?
> Here I've tried to support staticmethods inside bindings type:
> https://github.com/vitek/cython/commit/c0725ab340a8173d8e6724c62be3a135df58980e
>
>>
>> I think that small speed regression for better compatibility is OK if
>> we add a directive to not create binding functions. (It'd be nice if
>> we could work around it, but that's hard as CPython has special
>> hooks...) The bigger issue is that the binding behavior is backwards
>> incompatible, and perhaps in a subtle way. Perhaps we need to have a
>> phase where all currently non-binding functions that will become
>> binding functions will raise an error (or at least a warning) to wean
>> people off of the old behavior before making a switch.
>>
>
> Is the difference so significant?
If anyone is assigning a Cython function to an object and then using
it they're counting on the current non-binding behavior, and it will
break. The speed is probably a lesser issue, which is what benchmarks
are for.
- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Fused types syntax
In looking at merging fused types, it's time to nail down the syntax. The current implementation is ctypedef cython.fused_type(list, dict, object) fused_t This requires an addition to the grammer to allow the "call" syntax in a type declaration, as well as special casing to make it allowed only in a typedef. What about cython.fused_type[list, dict, object]. One advantage is that indexing is already valid in type declarations, and its the typical syntax for parameterized types. Thoughts? Any other ideas? - Robert P.S. Anyone remember buffers and C++ templated types are dissallowed as typedefs? ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused types syntax
Not that my opinion carriers any weight, but I'm +1 on this. On Thu, Jun 2, 2011 at 11:39 AM, Robert Bradshaw < rober...@math.washington.edu> wrote: > In looking at merging fused types, it's time to nail down the syntax. > The current implementation is > >ctypedef cython.fused_type(list, dict, object) fused_t > > This requires an addition to the grammer to allow the "call" syntax in > a type declaration, as well as special casing to make it allowed only > in a typedef. What about > >cython.fused_type[list, dict, object]. > > One advantage is that indexing is already valid in type declarations, > and its the typical syntax for parameterized types. Thoughts? Any > other ideas? > > - Robert > > > P.S. Anyone remember buffers and C++ templated types are dissallowed > as typedefs? > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On 2 June 2011 18:31, Robert Bradshaw wrote:
> On Thu, Jun 2, 2011 at 1:30 AM, Vitja Makarov wrote:
>> 2011/6/2 Robert Bradshaw :
>>> On Wed, Jun 1, 2011 at 7:26 AM, Vitja Makarov
>>> wrote:
2011/6/1 mark florisson :
> On 31 May 2011 20:25, Vitja Makarov wrote:
>> Hi!
>>
>> Is bindings performance issue valuable?
>>
>> $ cat bindbench.pyx
>> def wo_bindings():
>> pass
>>
>> def outer():
>> def inner():
>> pass
>> return inner
>> with_bindings = outer()
>>
>> $ python
> import timeit
> timeit.repeat('with_bindings()', setup='from bindbench import
> wo_bindings, with_bindings', repeat=1, number=1)
>> [6.169871807098389]
> timeit.repeat('wo_bindings()', setup='from bindbench import
> wo_bindings, with_bindings', repeat=1, number=1)
>> [4.609416961669922]
>>
>> PyCBindings makes it 1.3 (difference is about 15ns on my laptop) times
>> slower for CPython interpreter execution.
>> As CPython has some optimizations for CFunctions and PyCFunctions.
>>
>> Does it make sense for us? Or we can easily switch to bindings?
>>
>> --
>> vitja.
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>
> I think switching should be fine, if you'd desperately need the speed
> you'd be calling c(p)def functions from Cython. In fact, when the
> fused cfunction will be ready it will be even slightly slower, as it
> overrides the tp_call. But perhaps that should really be made into a
> subclass...
>>>
>>> Yes, this should be a subclass, slowing all calls down is a bad idea.
>>>
> Anyway, would you use these for Python classes and module-level def
> functions (and closures) only, or would you also use them in extension
> classes? Because I found at least a bit of problem there, as extension
> class methods get 'self' passed as the first argument to the C
> function (as PyCFunctionObject.m_self), whereas methods in Python
> classes get 'self' passed in the args tuple (and the m_self is
> unused).
>
Recently I've found a problem with static methods (__new__ for
example) of inner classes (classes created inside a function, for
instance)
I think binding version should be used for all regular def functions
(defs, staticmethods, classmethods and so on).
>>>
>>> +1 to moving that way. (Well, static and class methods bind
>>> differently, right?)
>>>
I also think that it's better to rename binding_PyCFunction_Type to
something like CyFunctionType and make it more like PyFunction.
>>>
>>> +1
>>>
>>> Is there any advantage to the method descriptors (other than that
>>> they're easier for a human to write?)
>>
>> Initially bindings was written to support bound class methods (am I right?)
>> So when we use it for regular functions 'binding' in the name doesn't
>> reflect its purpose.
>
> There's three kinds of functions we create: PyCFunctions, binding
> PyCFunctions, and extension class methods (via descriptors). I was
> asking about the latter.
Yeah those descriptors have no advantage at all, it's simple
convenience. So we could just use our binding descriptor. The only
difference is that those descriptors return PyCFunctionObjects (which
may have special-cased performance-related advantages, at Vitja
demonstrated), and we would return a binding function (that would have
the same dict as the original).
I still believe that you'd have to do something different for
extension classes (e.g. depending on a flag pass 'self' as m_self or
in the args tuple, so tp_call would have to be overridden). Besides,
if you're going to return a binding pycfunction from descr_get, you'll
want to do type checking for 'self' for unbound method calls. I got
all that working for normal python classes here:
https://github.com/markflorisson88/cython/tree/fusedtypes . Currently
it is intermixed with support for fused Python functions, but seeing
you'd override tp_call anyway the overhead should be negligible.
> As for the first two, it reflects the difference in behavior. If I
> take the function and assign it to a class, it will bind as a method
> when I go to look it up. This is arguably the biggest difference
> between built-in functions and Python functions.
>
>> One the other hand it's much more easy to write manually.
>>
>> About staticmethods: I think that CyFunction type should handle it as well
>> because staticmethods can have custom attributes and act just like
>> normal def one.
>>
>> The difference is insde descr_get it should return bound method for
>> normal and self for staticmethods.
>
> Yep. Does that merit a new class, or a flag that's checked on every
> function invocation (false 99.9% of the time)?
Well, it only needs to do the chec
Re: [Cython] Bindings performance issue
2011/6/2 Robert Bradshaw : >> Initially bindings was written to support bound class methods (am I right?) >> So when we use it for regular functions 'binding' in the name doesn't >> reflect its purpose. > > There's three kinds of functions we create: PyCFunctions, binding > PyCFunctions, and extension class methods (via descriptors). I was > asking about the latter. > > As for the first two, it reflects the difference in behavior. If I > take the function and assign it to a class, it will bind as a method > when I go to look it up. This is arguably the biggest difference > between built-in functions and Python functions. > >> One the other hand it's much more easy to write manually. >> >> About staticmethods: I think that CyFunction type should handle it as well >> because staticmethods can have custom attributes and act just like >> normal def one. >> >> The difference is insde descr_get it should return bound method for >> normal and self for staticmethods. > > Yep. Does that merit a new class, or a flag that's checked on every > function invocation (false 99.9% of the time)? > tp_descr_get() is called each time function gets bound, isn't so? If it makes sens we can create both CyFunction and CyStaticFunction types. >> Here I've tried to support staticmethods inside bindings type: >> https://github.com/vitek/cython/commit/c0725ab340a8173d8e6724c62be3a135df58980e >> >>> >>> I think that small speed regression for better compatibility is OK if >>> we add a directive to not create binding functions. (It'd be nice if >>> we could work around it, but that's hard as CPython has special >>> hooks...) The bigger issue is that the binding behavior is backwards >>> incompatible, and perhaps in a subtle way. Perhaps we need to have a >>> phase where all currently non-binding functions that will become >>> binding functions will raise an error (or at least a warning) to wean >>> people off of the old behavior before making a switch. >>> >> >> Is the difference so significant? > > If anyone is assigning a Cython function to an object and then using > it they're counting on the current non-binding behavior, and it will > break. The speed is probably a lesser issue, which is what benchmarks > are for. > -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused types syntax
On 2 June 2011 18:39, Robert Bradshaw wrote: > In looking at merging fused types, it's time to nail down the syntax. > The current implementation is > > ctypedef cython.fused_type(list, dict, object) fused_t > > This requires an addition to the grammer to allow the "call" syntax in > a type declaration, as well as special casing to make it allowed only > in a typedef. What about > > cython.fused_type[list, dict, object]. > > One advantage is that indexing is already valid in type declarations, > and its the typical syntax for parameterized types. Thoughts? Any > other ideas? I like it, so I'll give another +1. Changing wouldn't be too much work anyways. > - Robert > > > P.S. Anyone remember buffers and C++ templated types are dissallowed > as typedefs? Yes, there's even a test for it: tests/errors/buffertypedef_T117.pyx (http://trac.cython.org/cython_trac/ticket/117). But it looks more like an easy fix than a feature. > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On Thu, Jun 2, 2011 at 1:00 PM, mark florisson
wrote:
> On 2 June 2011 18:31, Robert Bradshaw wrote:
>> On Thu, Jun 2, 2011 at 1:30 AM, Vitja Makarov
>> wrote:
>>> 2011/6/2 Robert Bradshaw :
On Wed, Jun 1, 2011 at 7:26 AM, Vitja Makarov
wrote:
> 2011/6/1 mark florisson :
>> On 31 May 2011 20:25, Vitja Makarov wrote:
>>> Hi!
>>>
>>> Is bindings performance issue valuable?
>>>
>>> $ cat bindbench.pyx
>>> def wo_bindings():
>>> pass
>>>
>>> def outer():
>>> def inner():
>>> pass
>>> return inner
>>> with_bindings = outer()
>>>
>>> $ python
>> import timeit
>> timeit.repeat('with_bindings()', setup='from bindbench import
>> wo_bindings, with_bindings', repeat=1, number=1)
>>> [6.169871807098389]
>> timeit.repeat('wo_bindings()', setup='from bindbench import
>> wo_bindings, with_bindings', repeat=1, number=1)
>>> [4.609416961669922]
>>>
>>> PyCBindings makes it 1.3 (difference is about 15ns on my laptop) times
>>> slower for CPython interpreter execution.
>>> As CPython has some optimizations for CFunctions and PyCFunctions.
>>>
>>> Does it make sense for us? Or we can easily switch to bindings?
>>>
>>> --
>>> vitja.
>>> ___
>>> cython-devel mailing list
>>> cython-devel@python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>
>> I think switching should be fine, if you'd desperately need the speed
>> you'd be calling c(p)def functions from Cython. In fact, when the
>> fused cfunction will be ready it will be even slightly slower, as it
>> overrides the tp_call. But perhaps that should really be made into a
>> subclass...
Yes, this should be a subclass, slowing all calls down is a bad idea.
>> Anyway, would you use these for Python classes and module-level def
>> functions (and closures) only, or would you also use them in extension
>> classes? Because I found at least a bit of problem there, as extension
>> class methods get 'self' passed as the first argument to the C
>> function (as PyCFunctionObject.m_self), whereas methods in Python
>> classes get 'self' passed in the args tuple (and the m_self is
>> unused).
>>
>
> Recently I've found a problem with static methods (__new__ for
> example) of inner classes (classes created inside a function, for
> instance)
> I think binding version should be used for all regular def functions
> (defs, staticmethods, classmethods and so on).
+1 to moving that way. (Well, static and class methods bind
differently, right?)
> I also think that it's better to rename binding_PyCFunction_Type to
> something like CyFunctionType and make it more like PyFunction.
+1
Is there any advantage to the method descriptors (other than that
they're easier for a human to write?)
>>>
>>> Initially bindings was written to support bound class methods (am I right?)
>>> So when we use it for regular functions 'binding' in the name doesn't
>>> reflect its purpose.
>>
>> There's three kinds of functions we create: PyCFunctions, binding
>> PyCFunctions, and extension class methods (via descriptors). I was
>> asking about the latter.
>
> Yeah those descriptors have no advantage at all, it's simple
> convenience. So we could just use our binding descriptor. The only
> difference is that those descriptors return PyCFunctionObjects (which
> may have special-cased performance-related advantages, at Vitja
> demonstrated), and we would return a binding function (that would have
> the same dict as the original).
We could do this too on binding if we wanted...
> I still believe that you'd have to do something different for
> extension classes (e.g. depending on a flag pass 'self' as m_self or
> in the args tuple, so tp_call would have to be overridden). Besides,
> if you're going to return a binding pycfunction from descr_get, you'll
> want to do type checking for 'self' for unbound method calls. I got
> all that working for normal python classes here:
> https://github.com/markflorisson88/cython/tree/fusedtypes . Currently
> it is intermixed with support for fused Python functions, but seeing
> you'd override tp_call anyway the overhead should be negligible.
I was thinking of avoiding descriptors altogether, and simply
populating the cdef class dict.
>> As for the first two, it reflects the difference in behavior. If I
>> take the function and assign it to a class, it will bind as a method
>> when I go to look it up. This is arguably the biggest difference
>> between built-in functions and Python functions.
>>
>>> One the other hand it's much more easy to write manually.
>>>
>>> About staticmethods: I think that CyFunction type should handle it as well
>>> because staticmethods can
Re: [Cython] Fused types syntax
On Thu, Jun 2, 2011 at 1:07 PM, mark florisson wrote: > On 2 June 2011 18:39, Robert Bradshaw wrote: >> In looking at merging fused types, it's time to nail down the syntax. >> The current implementation is >> >> ctypedef cython.fused_type(list, dict, object) fused_t >> >> This requires an addition to the grammer to allow the "call" syntax in >> a type declaration, as well as special casing to make it allowed only >> in a typedef. What about >> >> cython.fused_type[list, dict, object]. >> >> One advantage is that indexing is already valid in type declarations, >> and its the typical syntax for parameterized types. Thoughts? Any >> other ideas? > > I like it, so I'll give another +1. Changing wouldn't be too much work > anyways. > >> - Robert >> >> >> P.S. Anyone remember buffers and C++ templated types are dissallowed >> as typedefs? > > Yes, there's even a test for it: tests/errors/buffertypedef_T117.pyx > (http://trac.cython.org/cython_trac/ticket/117). But it looks more > like an easy fix than a feature. I could see how for buffers it would require at bit more work, but for C++ types that shouldn't be a problem. In particular, I'm looking at https://github.com/cython/cython/blob/master/Cython/Compiler/Parsing.py#L2615 - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On 2 June 2011 22:16, Robert Bradshaw wrote:
> On Thu, Jun 2, 2011 at 1:00 PM, mark florisson
> wrote:
>> On 2 June 2011 18:31, Robert Bradshaw wrote:
>>> On Thu, Jun 2, 2011 at 1:30 AM, Vitja Makarov
>>> wrote:
2011/6/2 Robert Bradshaw :
> On Wed, Jun 1, 2011 at 7:26 AM, Vitja Makarov
> wrote:
>> 2011/6/1 mark florisson :
>>> On 31 May 2011 20:25, Vitja Makarov wrote:
Hi!
Is bindings performance issue valuable?
$ cat bindbench.pyx
def wo_bindings():
pass
def outer():
def inner():
pass
return inner
with_bindings = outer()
$ python
>>> import timeit
>>> timeit.repeat('with_bindings()', setup='from bindbench import
>>> wo_bindings, with_bindings', repeat=1, number=1)
[6.169871807098389]
>>> timeit.repeat('wo_bindings()', setup='from bindbench import
>>> wo_bindings, with_bindings', repeat=1, number=1)
[4.609416961669922]
PyCBindings makes it 1.3 (difference is about 15ns on my laptop) times
slower for CPython interpreter execution.
As CPython has some optimizations for CFunctions and PyCFunctions.
Does it make sense for us? Or we can easily switch to bindings?
--
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>> I think switching should be fine, if you'd desperately need the speed
>>> you'd be calling c(p)def functions from Cython. In fact, when the
>>> fused cfunction will be ready it will be even slightly slower, as it
>>> overrides the tp_call. But perhaps that should really be made into a
>>> subclass...
>
> Yes, this should be a subclass, slowing all calls down is a bad idea.
>
>>> Anyway, would you use these for Python classes and module-level def
>>> functions (and closures) only, or would you also use them in extension
>>> classes? Because I found at least a bit of problem there, as extension
>>> class methods get 'self' passed as the first argument to the C
>>> function (as PyCFunctionObject.m_self), whereas methods in Python
>>> classes get 'self' passed in the args tuple (and the m_self is
>>> unused).
>>>
>>
>> Recently I've found a problem with static methods (__new__ for
>> example) of inner classes (classes created inside a function, for
>> instance)
>> I think binding version should be used for all regular def functions
>> (defs, staticmethods, classmethods and so on).
>
> +1 to moving that way. (Well, static and class methods bind
> differently, right?)
>
>> I also think that it's better to rename binding_PyCFunction_Type to
>> something like CyFunctionType and make it more like PyFunction.
>
> +1
>
> Is there any advantage to the method descriptors (other than that
> they're easier for a human to write?)
Initially bindings was written to support bound class methods (am I right?)
So when we use it for regular functions 'binding' in the name doesn't
reflect its purpose.
>>>
>>> There's three kinds of functions we create: PyCFunctions, binding
>>> PyCFunctions, and extension class methods (via descriptors). I was
>>> asking about the latter.
>>
>> Yeah those descriptors have no advantage at all, it's simple
>> convenience. So we could just use our binding descriptor. The only
>> difference is that those descriptors return PyCFunctionObjects (which
>> may have special-cased performance-related advantages, at Vitja
>> demonstrated), and we would return a binding function (that would have
>> the same dict as the original).
>
> We could do this too on binding if we wanted...
That's what I mean, this all happens when we bind the function. If we
return a bound PyCFunctionObject, we lose our dict.
>> I still believe that you'd have to do something different for
>> extension classes (e.g. depending on a flag pass 'self' as m_self or
>> in the args tuple, so tp_call would have to be overridden). Besides,
>> if you're going to return a binding pycfunction from descr_get, you'll
>> want to do type checking for 'self' for unbound method calls. I got
>> all that working for normal python classes here:
>> https://github.com/markflorisson88/cython/tree/fusedtypes . Currently
>> it is intermixed with support for fused Python functions, but seeing
>> you'd override tp_call anyway the overhead should be negligible.
>
> I was thinking of avoiding descriptors altogether, and simply
> populating the cdef class dict.
Yes, that is my proposal. Without populating the class dict, you
cannot have custom pycfunctions.
>>> As for the first two, it reflects the difference in b
Re: [Cython] Bindings performance issue
On Thu, Jun 2, 2011 at 2:03 PM, mark florisson wrote: If anyone is assigning a Cython function to an object and then using it they're counting on the current non-binding behavior, and it will break. The speed is probably a lesser issue, which is what benchmarks are for. >>> >>> If you're binding functions to classes without expecting it to ever >>> bind, you don't really have bitching rights when stuff breaks later >>> on. You should have been using staticmethod() to begin with. And we >>> never said that our functions would never bind :) >> >> No, you're assigning it to an object, counting on being able to call >> it later on. E.g. the following is legal (though contrived in this >> example): >> >> sage: class A: >> : pass >> : >> sage: a = A() >> sage: a.foo = max >> sage: a.foo([1,2,3]) >> 3 >> >> If instead of len, it was one of our functions, then it would be bad >> to suddenly change the semantics, because it could still run but >> produce bad answers (e.g. if we had implemented max, suddenly a would >> be included in the comparison). This is why I proposed raising an >> explicit error as an intermediate step. >> >> If we don't promise anything, the contract is whatever the code does. >> That's the problem with not having a specification (which would be >> really nice, but is a lot of work). > > Functions on objects never get bound, they only get bound if they are > on the class. So your code would still work with binding functions. True. It would still break "A.foo = max" though. I'm not saying we should support or encourage this, but lets break it hard before we break it subtly. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused types syntax
On 06/02/2011 06:39 PM, Robert Bradshaw wrote: In looking at merging fused types, it's time to nail down the syntax. The current implementation is ctypedef cython.fused_type(list, dict, object) fused_t This requires an addition to the grammer to allow the "call" syntax in a type declaration, as well as special casing to make it allowed only in a typedef. What about cython.fused_type[list, dict, object]. One advantage is that indexing is already valid in type declarations, and its the typical syntax for parameterized types. Thoughts? Any other ideas? I don't really like overloading [] even more, and I think () (or, perhaps, 'fused_type([list, dict, object])'). But I don't feel very strongly about it. If you only want this allowed in typedefs, then, being puristic, I think that really a "fused type" is really different from a ctypedef, and that it would warrant something like a new keyword. cdef fusedtype [list, dict, object] fused_t That's rather horrible, but you get the gist. The thing is, once you use a ctypeudef, you really should allow def f(fused_type(float, double) x, fused_type(float, double) y): ... but then, how many fused types do you have, one or two? So this makes it seem to me that using ctypedef is a rather horrible hack. But, like I said, I don't feel strongly about this. P.S. Anyone remember buffers and C++ templated types are dissallowed as typedefs? As for buffers I just think I never got around to it... Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On 2 June 2011 23:13, Robert Bradshaw wrote: > On Thu, Jun 2, 2011 at 2:03 PM, mark florisson > wrote: > > If anyone is assigning a Cython function to an object and then using > it they're counting on the current non-binding behavior, and it will > break. The speed is probably a lesser issue, which is what benchmarks > are for. If you're binding functions to classes without expecting it to ever bind, you don't really have bitching rights when stuff breaks later on. You should have been using staticmethod() to begin with. And we never said that our functions would never bind :) >>> >>> No, you're assigning it to an object, counting on being able to call >>> it later on. E.g. the following is legal (though contrived in this >>> example): >>> >>> sage: class A: >>> : pass >>> : >>> sage: a = A() >>> sage: a.foo = max >>> sage: a.foo([1,2,3]) >>> 3 >>> >>> If instead of len, it was one of our functions, then it would be bad >>> to suddenly change the semantics, because it could still run but >>> produce bad answers (e.g. if we had implemented max, suddenly a would >>> be included in the comparison). This is why I proposed raising an >>> explicit error as an intermediate step. >>> >>> If we don't promise anything, the contract is whatever the code does. >>> That's the problem with not having a specification (which would be >>> really nice, but is a lot of work). >> >> Functions on objects never get bound, they only get bound if they are >> on the class. So your code would still work with binding functions. > > True. It would still break "A.foo = max" though. I'm not saying we > should support or encourage this, but lets break it hard before we > break it subtly. Again, such code is highly fragile and frankly incorrect to begin with, as it's based on the assumption that "Cython functions" never get bound. Getting functions (defined outside of class bodies) to bind in classes is a feature, I sometimes found myself to want it. So basically an error would be fine, but it would prevent normal usage as we have it in Python. > - Robert > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused types syntax
On Jun 2, 2011 2:18 PM, "Dag Sverre Seljebotn" wrote: > If you only want this allowed in typedefs, then, being puristic, I think that really a "fused type" is really different from a ctypedef, and that it would warrant something like a new keyword. > > cdef fusedtype [list, dict, object] fused_t +1 (to some variant of this, not necessarily this exactly) I find the argument that defining the same fused type twice results in two different, incompatible types to be very compelling. That's a fundamental difference from typedefs. - Nathaniel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused types syntax
On Thu, Jun 2, 2011 at 2:18 PM, Dag Sverre Seljebotn wrote: > On 06/02/2011 06:39 PM, Robert Bradshaw wrote: >> >> In looking at merging fused types, it's time to nail down the syntax. >> The current implementation is >> >> ctypedef cython.fused_type(list, dict, object) fused_t >> >> This requires an addition to the grammer to allow the "call" syntax in >> a type declaration, as well as special casing to make it allowed only >> in a typedef. What about >> >> cython.fused_type[list, dict, object]. >> >> One advantage is that indexing is already valid in type declarations, >> and its the typical syntax for parameterized types. Thoughts? Any >> other ideas? > > I don't really like overloading [] even more, and I think () (or, perhaps, > 'fused_type([list, dict, object])'). > > But I don't feel very strongly about it. > > If you only want this allowed in typedefs, then, being puristic, I think > that really a "fused type" is really different from a ctypedef, and that it > would warrant something like a new keyword. > > cdef fusedtype [list, dict, object] fused_t > > That's rather horrible, but you get the gist. The thing is, once you use a > ctypeudef, you really should allow > > def f(fused_type(float, double) x, fused_type(float, double) y): ... > > but then, how many fused types do you have, one or two? Two, and you can't refer to them. Such anonymous types could be handy for one-off functions, e.g. def f(fused_type[list, dict, object] x): ... but maybe that's not needed. > So this makes it seem to me that using ctypedef is a rather horrible hack. Yeah, this is the crux of the issue. (To be clear, Mark's implementation is good, it's our syntax that's hacky). It's perhaps less ugly than introducing a new keyword. A crazy idea: cdef fused floating_t: float double long double (in analogy with how we declare types like structs and enums). > But, like I said, I don't feel strongly about this. > > >> >> P.S. Anyone remember buffers and C++ templated types are dissallowed >> as typedefs? > > As for buffers I just think I never got around to it... And in that case you can't just punt the typedef to C :). - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On Thu, Jun 2, 2011 at 2:21 PM, mark florisson wrote: > On 2 June 2011 23:13, Robert Bradshaw wrote: >> On Thu, Jun 2, 2011 at 2:03 PM, mark florisson >> wrote: >> >> If anyone is assigning a Cython function to an object and then using >> it they're counting on the current non-binding behavior, and it will >> break. The speed is probably a lesser issue, which is what benchmarks >> are for. > > If you're binding functions to classes without expecting it to ever > bind, you don't really have bitching rights when stuff breaks later > on. You should have been using staticmethod() to begin with. And we > never said that our functions would never bind :) No, you're assigning it to an object, counting on being able to call it later on. E.g. the following is legal (though contrived in this example): sage: class A: : pass : sage: a = A() sage: a.foo = max sage: a.foo([1,2,3]) 3 If instead of len, it was one of our functions, then it would be bad to suddenly change the semantics, because it could still run but produce bad answers (e.g. if we had implemented max, suddenly a would be included in the comparison). This is why I proposed raising an explicit error as an intermediate step. If we don't promise anything, the contract is whatever the code does. That's the problem with not having a specification (which would be really nice, but is a lot of work). >>> >>> Functions on objects never get bound, they only get bound if they are >>> on the class. So your code would still work with binding functions. >> >> True. It would still break "A.foo = max" though. I'm not saying we >> should support or encourage this, but lets break it hard before we >> break it subtly. > > Again, such code is highly fragile and frankly incorrect to begin > with, as it's based on the assumption that "Cython functions" never > get bound. I agree, but I bet there's code out there depending on it, in particular workarounds for our current broken semantics will themselves break. > Getting functions (defined outside of class bodies) to bind > in classes is a feature, I sometimes found myself to want it. So > basically an error would be fine, but it would prevent normal usage as > we have it in Python. The error would just be for a transition period. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused types syntax
On 2 June 2011 23:31, Robert Bradshaw wrote: > On Thu, Jun 2, 2011 at 2:18 PM, Dag Sverre Seljebotn > wrote: >> On 06/02/2011 06:39 PM, Robert Bradshaw wrote: >>> >>> In looking at merging fused types, it's time to nail down the syntax. >>> The current implementation is >>> >>> ctypedef cython.fused_type(list, dict, object) fused_t >>> >>> This requires an addition to the grammer to allow the "call" syntax in >>> a type declaration, as well as special casing to make it allowed only >>> in a typedef. What about >>> >>> cython.fused_type[list, dict, object]. >>> >>> One advantage is that indexing is already valid in type declarations, >>> and its the typical syntax for parameterized types. Thoughts? Any >>> other ideas? >> >> I don't really like overloading [] even more, and I think () (or, perhaps, >> 'fused_type([list, dict, object])'). >> >> But I don't feel very strongly about it. >> >> If you only want this allowed in typedefs, then, being puristic, I think >> that really a "fused type" is really different from a ctypedef, and that it >> would warrant something like a new keyword. >> >> cdef fusedtype [list, dict, object] fused_t >> >> That's rather horrible, but you get the gist. The thing is, once you use a >> ctypeudef, you really should allow >> >> def f(fused_type(float, double) x, fused_type(float, double) y): ... >> >> but then, how many fused types do you have, one or two? > > Two, and you can't refer to them. Such anonymous types could be handy > for one-off functions, e.g. > > def f(fused_type[list, dict, object] x): ... > > but maybe that's not needed. > >> So this makes it seem to me that using ctypedef is a rather horrible hack. > > Yeah, this is the crux of the issue. (To be clear, Mark's > implementation is good, it's our syntax that's hacky). It's perhaps > less ugly than > introducing a new keyword. A crazy idea: > > cdef fused floating_t: > float > double > long double > > (in analogy with how we declare types like structs and enums). I like that syntax, the bad thing about ctypedef for me is that everything goes on one line, at which point you have to start breaking them if they get too long (which can happen quite easily with lenghty names like 'long double complex'), which looks awkward. So much for trying to reduce grammar changes, though :) >> But, like I said, I don't feel strongly about this. >> >> >>> >>> P.S. Anyone remember buffers and C++ templated types are dissallowed >>> as typedefs? >> >> As for buffers I just think I never got around to it... > > And in that case you can't just punt the typedef to C :). > > - Robert > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On 2 June 2011 23:34, Robert Bradshaw wrote: > On Thu, Jun 2, 2011 at 2:21 PM, mark florisson > wrote: >> On 2 June 2011 23:13, Robert Bradshaw wrote: >>> On Thu, Jun 2, 2011 at 2:03 PM, mark florisson >>> wrote: >>> >>> If anyone is assigning a Cython function to an object and then using >>> it they're counting on the current non-binding behavior, and it will >>> break. The speed is probably a lesser issue, which is what benchmarks >>> are for. >> >> If you're binding functions to classes without expecting it to ever >> bind, you don't really have bitching rights when stuff breaks later >> on. You should have been using staticmethod() to begin with. And we >> never said that our functions would never bind :) > > No, you're assigning it to an object, counting on being able to call > it later on. E.g. the following is legal (though contrived in this > example): > > sage: class A: > : pass > : > sage: a = A() > sage: a.foo = max > sage: a.foo([1,2,3]) > 3 > > If instead of len, it was one of our functions, then it would be bad > to suddenly change the semantics, because it could still run but > produce bad answers (e.g. if we had implemented max, suddenly a would > be included in the comparison). This is why I proposed raising an > explicit error as an intermediate step. > > If we don't promise anything, the contract is whatever the code does. > That's the problem with not having a specification (which would be > really nice, but is a lot of work). Functions on objects never get bound, they only get bound if they are on the class. So your code would still work with binding functions. >>> >>> True. It would still break "A.foo = max" though. I'm not saying we >>> should support or encourage this, but lets break it hard before we >>> break it subtly. >> >> Again, such code is highly fragile and frankly incorrect to begin >> with, as it's based on the assumption that "Cython functions" never >> get bound. > > I agree, but I bet there's code out there depending on it, in > particular workarounds for our current broken semantics will > themselves break. Workarounds wouldn't break, as they would wrap the non-binding function in another object, and implement __get__ to return a new object that, when called, would call the original function with 'self' as the first argument. >> Getting functions (defined outside of class bodies) to bind >> in classes is a feature, I sometimes found myself to want it. So >> basically an error would be fine, but it would prevent normal usage as >> we have it in Python. > > The error would just be for a transition period. The transition period would be for an entire release? > - Robert > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused types syntax
On Thu, Jun 2, 2011 at 2:42 PM, mark florisson wrote: > On 2 June 2011 23:31, Robert Bradshaw wrote: >> On Thu, Jun 2, 2011 at 2:18 PM, Dag Sverre Seljebotn >> wrote: >>> On 06/02/2011 06:39 PM, Robert Bradshaw wrote: In looking at merging fused types, it's time to nail down the syntax. The current implementation is ctypedef cython.fused_type(list, dict, object) fused_t This requires an addition to the grammer to allow the "call" syntax in a type declaration, as well as special casing to make it allowed only in a typedef. What about cython.fused_type[list, dict, object]. One advantage is that indexing is already valid in type declarations, and its the typical syntax for parameterized types. Thoughts? Any other ideas? >>> >>> I don't really like overloading [] even more, and I think () (or, perhaps, >>> 'fused_type([list, dict, object])'). >>> >>> But I don't feel very strongly about it. >>> >>> If you only want this allowed in typedefs, then, being puristic, I think >>> that really a "fused type" is really different from a ctypedef, and that it >>> would warrant something like a new keyword. >>> >>> cdef fusedtype [list, dict, object] fused_t >>> >>> That's rather horrible, but you get the gist. The thing is, once you use a >>> ctypeudef, you really should allow >>> >>> def f(fused_type(float, double) x, fused_type(float, double) y): ... >>> >>> but then, how many fused types do you have, one or two? >> >> Two, and you can't refer to them. Such anonymous types could be handy >> for one-off functions, e.g. >> >> def f(fused_type[list, dict, object] x): ... >> >> but maybe that's not needed. >> >>> So this makes it seem to me that using ctypedef is a rather horrible hack. >> >> Yeah, this is the crux of the issue. (To be clear, Mark's >> implementation is good, it's our syntax that's hacky). It's perhaps >> less ugly than >> introducing a new keyword. A crazy idea: >> >> cdef fused floating_t: >> float >> double >> long double >> >> (in analogy with how we declare types like structs and enums). > > I like that syntax, the bad thing about ctypedef for me is that > everything goes on one line, at which point you have to start breaking > them if they get too long (which can happen quite easily with lenghty > names like 'long double complex'), which looks awkward. > > So much for trying to reduce grammar changes, though :) Well, it introduces a new kind of node, rather than the kind of funny interaction/restriction with typedef, which got me thinking about this in the first place. Of course then you can't create an anonymous one for use in a function (which may be viewed as a feature or a defect). - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On Thu, Jun 2, 2011 at 2:45 PM, mark florisson wrote: > On 2 June 2011 23:34, Robert Bradshaw wrote: >> On Thu, Jun 2, 2011 at 2:21 PM, mark florisson >> wrote: >>> On 2 June 2011 23:13, Robert Bradshaw wrote: On Thu, Jun 2, 2011 at 2:03 PM, mark florisson wrote: If anyone is assigning a Cython function to an object and then using it they're counting on the current non-binding behavior, and it will break. The speed is probably a lesser issue, which is what benchmarks are for. >>> >>> If you're binding functions to classes without expecting it to ever >>> bind, you don't really have bitching rights when stuff breaks later >>> on. You should have been using staticmethod() to begin with. And we >>> never said that our functions would never bind :) >> >> No, you're assigning it to an object, counting on being able to call >> it later on. E.g. the following is legal (though contrived in this >> example): >> >> sage: class A: >> : pass >> : >> sage: a = A() >> sage: a.foo = max >> sage: a.foo([1,2,3]) >> 3 >> >> If instead of len, it was one of our functions, then it would be bad >> to suddenly change the semantics, because it could still run but >> produce bad answers (e.g. if we had implemented max, suddenly a would >> be included in the comparison). This is why I proposed raising an >> explicit error as an intermediate step. >> >> If we don't promise anything, the contract is whatever the code does. >> That's the problem with not having a specification (which would be >> really nice, but is a lot of work). > > Functions on objects never get bound, they only get bound if they are > on the class. So your code would still work with binding functions. True. It would still break "A.foo = max" though. I'm not saying we should support or encourage this, but lets break it hard before we break it subtly. >>> >>> Again, such code is highly fragile and frankly incorrect to begin >>> with, as it's based on the assumption that "Cython functions" never >>> get bound. >> >> I agree, but I bet there's code out there depending on it, in >> particular workarounds for our current broken semantics will >> themselves break. > > Workarounds wouldn't break, as they would wrap the non-binding > function in another object, and implement __get__ to return a new > object that, when called, would call the original function with 'self' > as the first argument. Depends on the workaround. I'm thinking workarounds like "oh, self isn't getting passed, guess I'll have to pass it manually here..." >>> Getting functions (defined outside of class bodies) to bind >>> in classes is a feature, I sometimes found myself to want it. So >>> basically an error would be fine, but it would prevent normal usage as >>> we have it in Python. >> >> The error would just be for a transition period. > > The transition period would be for an entire release? Yes, though hopefully a much shorter release cycle than this last one. I just haven't had time to fix those remaining failing Sage tests, and we keep merging in more and more stuff (which is good, but we're well overdue for a release by now.) - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On 2 June 2011 23:59, Robert Bradshaw wrote: > On Thu, Jun 2, 2011 at 2:45 PM, mark florisson > wrote: >> On 2 June 2011 23:34, Robert Bradshaw wrote: >>> On Thu, Jun 2, 2011 at 2:21 PM, mark florisson >>> wrote: On 2 June 2011 23:13, Robert Bradshaw wrote: > On Thu, Jun 2, 2011 at 2:03 PM, mark florisson > wrote: > > If anyone is assigning a Cython function to an object and then using > it they're counting on the current non-binding behavior, and it will > break. The speed is probably a lesser issue, which is what benchmarks > are for. If you're binding functions to classes without expecting it to ever bind, you don't really have bitching rights when stuff breaks later on. You should have been using staticmethod() to begin with. And we never said that our functions would never bind :) >>> >>> No, you're assigning it to an object, counting on being able to call >>> it later on. E.g. the following is legal (though contrived in this >>> example): >>> >>> sage: class A: >>> : pass >>> : >>> sage: a = A() >>> sage: a.foo = max >>> sage: a.foo([1,2,3]) >>> 3 >>> >>> If instead of len, it was one of our functions, then it would be bad >>> to suddenly change the semantics, because it could still run but >>> produce bad answers (e.g. if we had implemented max, suddenly a would >>> be included in the comparison). This is why I proposed raising an >>> explicit error as an intermediate step. >>> >>> If we don't promise anything, the contract is whatever the code does. >>> That's the problem with not having a specification (which would be >>> really nice, but is a lot of work). >> >> Functions on objects never get bound, they only get bound if they are >> on the class. So your code would still work with binding functions. > > True. It would still break "A.foo = max" though. I'm not saying we > should support or encourage this, but lets break it hard before we > break it subtly. Again, such code is highly fragile and frankly incorrect to begin with, as it's based on the assumption that "Cython functions" never get bound. >>> >>> I agree, but I bet there's code out there depending on it, in >>> particular workarounds for our current broken semantics will >>> themselves break. >> >> Workarounds wouldn't break, as they would wrap the non-binding >> function in another object, and implement __get__ to return a new >> object that, when called, would call the original function with 'self' >> as the first argument. > > Depends on the workaround. I'm thinking workarounds like "oh, self > isn't getting passed, guess I'll have to pass it manually here..." Ah, something like functools.partial. Yeah, that'd break :) Getting functions (defined outside of class bodies) to bind in classes is a feature, I sometimes found myself to want it. So basically an error would be fine, but it would prevent normal usage as we have it in Python. >>> >>> The error would just be for a transition period. >> >> The transition period would be for an entire release? > > Yes, though hopefully a much shorter release cycle than this last one. > I just haven't had time to fix those remaining failing Sage tests, and > we keep merging in more and more stuff (which is good, but we're well > overdue for a release by now.) Ok. Then perhaps it would be better to merge the support for fused types in the next release? I won't be available until the 6th of July for coding, so until then we'd be stuck with cython.fused_type() in a ctypedef. > - Robert > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On 3 June 2011 00:04, mark florisson wrote: > On 2 June 2011 23:59, Robert Bradshaw wrote: >> On Thu, Jun 2, 2011 at 2:45 PM, mark florisson >> wrote: >>> On 2 June 2011 23:34, Robert Bradshaw wrote: On Thu, Jun 2, 2011 at 2:21 PM, mark florisson wrote: > On 2 June 2011 23:13, Robert Bradshaw > wrote: >> On Thu, Jun 2, 2011 at 2:03 PM, mark florisson >> wrote: >> >> If anyone is assigning a Cython function to an object and then using >> it they're counting on the current non-binding behavior, and it will >> break. The speed is probably a lesser issue, which is what benchmarks >> are for. > > If you're binding functions to classes without expecting it to ever > bind, you don't really have bitching rights when stuff breaks later > on. You should have been using staticmethod() to begin with. And we > never said that our functions would never bind :) No, you're assigning it to an object, counting on being able to call it later on. E.g. the following is legal (though contrived in this example): sage: class A: : pass : sage: a = A() sage: a.foo = max sage: a.foo([1,2,3]) 3 If instead of len, it was one of our functions, then it would be bad to suddenly change the semantics, because it could still run but produce bad answers (e.g. if we had implemented max, suddenly a would be included in the comparison). This is why I proposed raising an explicit error as an intermediate step. If we don't promise anything, the contract is whatever the code does. That's the problem with not having a specification (which would be really nice, but is a lot of work). >>> >>> Functions on objects never get bound, they only get bound if they are >>> on the class. So your code would still work with binding functions. >> >> True. It would still break "A.foo = max" though. I'm not saying we >> should support or encourage this, but lets break it hard before we >> break it subtly. > > Again, such code is highly fragile and frankly incorrect to begin > with, as it's based on the assumption that "Cython functions" never > get bound. I agree, but I bet there's code out there depending on it, in particular workarounds for our current broken semantics will themselves break. >>> >>> Workarounds wouldn't break, as they would wrap the non-binding >>> function in another object, and implement __get__ to return a new >>> object that, when called, would call the original function with 'self' >>> as the first argument. >> >> Depends on the workaround. I'm thinking workarounds like "oh, self >> isn't getting passed, guess I'll have to pass it manually here..." > > Ah, something like functools.partial. Yeah, that'd break :) > > Getting functions (defined outside of class bodies) to bind > in classes is a feature, I sometimes found myself to want it. So > basically an error would be fine, but it would prevent normal usage as > we have it in Python. The error would just be for a transition period. >>> >>> The transition period would be for an entire release? >> >> Yes, though hopefully a much shorter release cycle than this last one. >> I just haven't had time to fix those remaining failing Sage tests, and >> we keep merging in more and more stuff (which is good, but we're well >> overdue for a release by now.) > > Ok. Then perhaps it would be better to merge the support for fused > types in the next release? I won't be available until the 6th of July > for coding, so until then we'd be stuck with cython.fused_type() in a > ctypedef. ... where 'next' means not the upcoming release, but the one after that. >> - Robert >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel >> > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bindings performance issue
On Thu, Jun 2, 2011 at 3:04 PM, mark florisson wrote: > On 2 June 2011 23:59, Robert Bradshaw wrote: >> On Thu, Jun 2, 2011 at 2:45 PM, mark florisson >> wrote: >>> On 2 June 2011 23:34, Robert Bradshaw wrote: On Thu, Jun 2, 2011 at 2:21 PM, mark florisson wrote: > On 2 June 2011 23:13, Robert Bradshaw > wrote: >> On Thu, Jun 2, 2011 at 2:03 PM, mark florisson >> wrote: >> >> If anyone is assigning a Cython function to an object and then using >> it they're counting on the current non-binding behavior, and it will >> break. The speed is probably a lesser issue, which is what benchmarks >> are for. > > If you're binding functions to classes without expecting it to ever > bind, you don't really have bitching rights when stuff breaks later > on. You should have been using staticmethod() to begin with. And we > never said that our functions would never bind :) No, you're assigning it to an object, counting on being able to call it later on. E.g. the following is legal (though contrived in this example): sage: class A: : pass : sage: a = A() sage: a.foo = max sage: a.foo([1,2,3]) 3 If instead of len, it was one of our functions, then it would be bad to suddenly change the semantics, because it could still run but produce bad answers (e.g. if we had implemented max, suddenly a would be included in the comparison). This is why I proposed raising an explicit error as an intermediate step. If we don't promise anything, the contract is whatever the code does. That's the problem with not having a specification (which would be really nice, but is a lot of work). >>> >>> Functions on objects never get bound, they only get bound if they are >>> on the class. So your code would still work with binding functions. >> >> True. It would still break "A.foo = max" though. I'm not saying we >> should support or encourage this, but lets break it hard before we >> break it subtly. > > Again, such code is highly fragile and frankly incorrect to begin > with, as it's based on the assumption that "Cython functions" never > get bound. I agree, but I bet there's code out there depending on it, in particular workarounds for our current broken semantics will themselves break. >>> >>> Workarounds wouldn't break, as they would wrap the non-binding >>> function in another object, and implement __get__ to return a new >>> object that, when called, would call the original function with 'self' >>> as the first argument. >> >> Depends on the workaround. I'm thinking workarounds like "oh, self >> isn't getting passed, guess I'll have to pass it manually here..." > > Ah, something like functools.partial. Yeah, that'd break :) > > Getting functions (defined outside of class bodies) to bind > in classes is a feature, I sometimes found myself to want it. So > basically an error would be fine, but it would prevent normal usage as > we have it in Python. The error would just be for a transition period. >>> >>> The transition period would be for an entire release? >> >> Yes, though hopefully a much shorter release cycle than this last one. >> I just haven't had time to fix those remaining failing Sage tests, and >> we keep merging in more and more stuff (which is good, but we're well >> overdue for a release by now.) > > Ok. Then perhaps it would be better to merge the support for fused > types in the next release? I won't be available until the 6th of July > for coding, so until then we'd be stuck with cython.fused_type() in a > ctypedef. Well, it's such a cool feature :). More to the point, it's not nearly as invasive (i.e. compiling existing code will still pretty much work the way it always has). - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
