Re: [Cython] weird declarations in fused types C code
On 11 May 2012 07:38, Stefan Behnel wrote: > Hi, > > while trying to replace the "import sys; if sys.version_info >= (3,0)" in > the fused types dispatch code by the more straight forward "if > PY_MAJOR_VERSION >= 3" (before I came to think that this particular case > only guards useless code that does the wrong thing), Yes, you made that plenty clear, sorry for thinking in terms of python code. For the record, it does do the right thing. > I noticed that the > code generates a declaration of PyErr_Clear() into the outside environment. > When used in cdef classes, this leads to an external method being declared > in the class, essentially like this: > > cdef class MyClass: > cdef extern from *: > void PyErr_Clear() > > Surprisingly enough, this actually works. Cython assigns the real C-API > function pointer to it during type initialisation and even calls the > function directly (instead of going through the vtab) when used. A rather > curious feature that I would never had thought of. Yes, normally the parser catches that. > Anyway, this side effect is obviously a bug in the fused types dispatch, > but I don't have a good idea on how to fix it. I'm sure Mark put some > thought into this while trying hard to make it work and just didn't notice > the impact on type namespaces. I am aware of this behaviour, the thing is that the dispatcher function needs to be analyzed in the right context in order to generate an appropriate function or method in case of a cdef class (which are different from methods in normal classes even when synthesized). I thought about splitting the declarations from the actual function, and analyzing that in the module scope. Perhaps with some name mangling this can avoid names being accidentally available in user code. I don't recall if I have tried that already, but I'll give it another try. > I've put up a pull request to remove the Py3 specialisation code, but this > is worth some more consideration. Looks good to me, I'll merge it. > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] weird declarations in fused types C code
On 11 May 2012 11:44, mark florisson wrote: > On 11 May 2012 07:38, Stefan Behnel wrote: >> Hi, >> >> while trying to replace the "import sys; if sys.version_info >= (3,0)" in >> the fused types dispatch code by the more straight forward "if >> PY_MAJOR_VERSION >= 3" (before I came to think that this particular case >> only guards useless code that does the wrong thing), > > Yes, you made that plenty clear, sorry for thinking in terms of python > code. For the record, it does do the right thing. > >> I noticed that the >> code generates a declaration of PyErr_Clear() into the outside environment. >> When used in cdef classes, this leads to an external method being declared >> in the class, essentially like this: >> >> cdef class MyClass: >> cdef extern from *: >> void PyErr_Clear() >> >> Surprisingly enough, this actually works. Cython assigns the real C-API >> function pointer to it during type initialisation and even calls the >> function directly (instead of going through the vtab) when used. A rather >> curious feature that I would never had thought of. > > Yes, normally the parser catches that. > >> Anyway, this side effect is obviously a bug in the fused types dispatch, >> but I don't have a good idea on how to fix it. I'm sure Mark put some >> thought into this while trying hard to make it work and just didn't notice >> the impact on type namespaces. > > I am aware of this behaviour, the thing is that the dispatcher > function needs to be analyzed in the right context in order to > generate an appropriate function or method in case of a cdef class > (which are different from methods in normal classes even when > synthesized). I thought about splitting the declarations from the > actual function, and analyzing that in the module scope. Perhaps with > some name mangling this can avoid names being accidentally available > in user code. I don't recall if I have tried that already, but I'll > give it another try. Ah, I see I already split them, all that is needed is to put it in the global scope now :) >> I've put up a pull request to remove the Py3 specialisation code, but this >> is worth some more consideration. > > Looks good to me, I'll merge it. > >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] weird declarations in fused types C code
On 11 May 2012 11:54, mark florisson wrote: > On 11 May 2012 11:44, mark florisson wrote: >> On 11 May 2012 07:38, Stefan Behnel wrote: >>> Hi, >>> >>> while trying to replace the "import sys; if sys.version_info >= (3,0)" in >>> the fused types dispatch code by the more straight forward "if >>> PY_MAJOR_VERSION >= 3" (before I came to think that this particular case >>> only guards useless code that does the wrong thing), >> >> Yes, you made that plenty clear, sorry for thinking in terms of python >> code. For the record, it does do the right thing. >> >>> I noticed that the >>> code generates a declaration of PyErr_Clear() into the outside environment. >>> When used in cdef classes, this leads to an external method being declared >>> in the class, essentially like this: >>> >>> cdef class MyClass: >>> cdef extern from *: >>> void PyErr_Clear() >>> >>> Surprisingly enough, this actually works. Cython assigns the real C-API >>> function pointer to it during type initialisation and even calls the >>> function directly (instead of going through the vtab) when used. A rather >>> curious feature that I would never had thought of. >> >> Yes, normally the parser catches that. >> >>> Anyway, this side effect is obviously a bug in the fused types dispatch, >>> but I don't have a good idea on how to fix it. I'm sure Mark put some >>> thought into this while trying hard to make it work and just didn't notice >>> the impact on type namespaces. >> >> I am aware of this behaviour, the thing is that the dispatcher >> function needs to be analyzed in the right context in order to >> generate an appropriate function or method in case of a cdef class >> (which are different from methods in normal classes even when >> synthesized). I thought about splitting the declarations from the >> actual function, and analyzing that in the module scope. Perhaps with >> some name mangling this can avoid names being accidentally available >> in user code. I don't recall if I have tried that already, but I'll >> give it another try. > > Ah, I see I already split them, all that is needed is to put it in the > global scope now :) https://github.com/markflorisson88/cython/commit/3500fcd01ce6e68e76fcbabfe009eb273d7972fb >>> I've put up a pull request to remove the Py3 specialisation code, but this >>> is worth some more consideration. >> >> Looks good to me, I'll merge it. >> >>> Stefan >>> ___ >>> cython-devel mailing list >>> cython-devel@python.org >>> http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] weird declarations in fused types C code
mark florisson, 11.05.2012 13:00: > On 11 May 2012 11:54, mark florisson wrote: >> On 11 May 2012 11:44, mark florisson wrote: >>> On 11 May 2012 07:38, Stefan Behnel wrote: while trying to replace the "import sys; if sys.version_info >= (3,0)" in the fused types dispatch code by the more straight forward "if PY_MAJOR_VERSION >= 3" (before I came to think that this particular case only guards useless code that does the wrong thing), >>> >>> Yes, you made that plenty clear, sorry for thinking in terms of python >>> code. For the record, it does do the right thing. >>> I noticed that the code generates a declaration of PyErr_Clear() into the outside environment. When used in cdef classes, this leads to an external method being declared in the class, essentially like this: cdef class MyClass: cdef extern from *: void PyErr_Clear() Surprisingly enough, this actually works. Cython assigns the real C-API function pointer to it during type initialisation and even calls the function directly (instead of going through the vtab) when used. A rather curious feature that I would never had thought of. >>> >>> Yes, normally the parser catches that. >>> Anyway, this side effect is obviously a bug in the fused types dispatch, but I don't have a good idea on how to fix it. I'm sure Mark put some thought into this while trying hard to make it work and just didn't notice the impact on type namespaces. >>> >>> I am aware of this behaviour, the thing is that the dispatcher >>> function needs to be analyzed in the right context in order to >>> generate an appropriate function or method in case of a cdef class >>> (which are different from methods in normal classes even when >>> synthesized). I thought about splitting the declarations from the >>> actual function, and analyzing that in the module scope. Perhaps with >>> some name mangling this can avoid names being accidentally available >>> in user code. I don't recall if I have tried that already, but I'll >>> give it another try. >> >> Ah, I see I already split them, all that is needed is to put it in the >> global scope now :) > > https://github.com/markflorisson88/cython/commit/3500fcd01ce6e68e76fcbabfe009eb273d7972fb Ok, sure, works for me. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] CEP 1001 - Custom PyTypeObject extensions
This comes from a refactor of the work on CEP 1000: A PEP proposal, with a hack for use in current Python versions and in the case of PEP rejection, that allows 3rd party libraries to agree on extensions to PyTypeObject. http://wiki.cython.org/enhancements/cep1001 If this makes it as a PEP, I don't think we need to think about having CEP 1000 accepted as a PEP. Comments? Dag ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] CF based type inference
On Tue, May 8, 2012 at 11:22 PM, Stefan Behnel wrote:
> Robert Bradshaw, 09.05.2012 00:12:
>> On Tue, May 8, 2012 at 6:47 AM, Vitja Makarov wrote:
>>> 2012/5/8 Stefan Behnel:
Vitja has rebased the type inference on the control flow, so I wonder if
this will enable us to properly infer this:
def partial_validity():
"""
>>> partial_validity()
('Python object', 'double', 'str object')
"""
a = 1.0
b = a + 2 # definitely double
a = 'test'
c = a + 'toast' # definitely str
return typeof(a), typeof(b), typeof(c)
I think, what is mainly needed for this is that a NameNode with an
undeclared type should not report its own entry as dependency but that of
its own cf_assignments. Would this work?
(Haven't got the time to try it out right now, so I'm dumping it here.)
>>>
>>> Yeah, that might work. The other way to go is to split entries:
>>>
>>> def partial_validity():
>>> """
>>> >>> partial_validity()
>>> ('str object', 'double', 'str object')
>>> """
>>> a_1 = 1.0
>>> b = a_1 + 2 # definitely double
>>> a_2 = 'test'
>>> c = a_2 + 'toast' # definitely str
>>> return typeof(a_2), typeof(b), typeof(c)
>>>
>>> And this should work better because it allows to infer a_1 as a double
>>> and a_2 as a string.
>>
>> This already works, right?
>
> It would work if it was implemented. *wink*
Well, we don't infer str, but that's a separate issue from control flow.
- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] CF based type inference
Robert Bradshaw, 11.05.2012 17:48:
> On Tue, May 8, 2012 at 11:22 PM, Stefan Behnel wrote:
>> Robert Bradshaw, 09.05.2012 00:12:
>>> On Tue, May 8, 2012 at 6:47 AM, Vitja Makarov wrote:
2012/5/8 Stefan Behnel:
> Vitja has rebased the type inference on the control flow, so I wonder if
> this will enable us to properly infer this:
>
> def partial_validity():
>"""
>>>> partial_validity()
>('Python object', 'double', 'str object')
>"""
>a = 1.0
>b = a + 2 # definitely double
>a = 'test'
>c = a + 'toast' # definitely str
>return typeof(a), typeof(b), typeof(c)
>
> I think, what is mainly needed for this is that a NameNode with an
> undeclared type should not report its own entry as dependency but that of
> its own cf_assignments. Would this work?
>
> (Haven't got the time to try it out right now, so I'm dumping it here.)
>
Yeah, that might work. The other way to go is to split entries:
def partial_validity():
"""
>>> partial_validity()
('str object', 'double', 'str object')
"""
a_1 = 1.0
b = a_1 + 2 # definitely double
a_2 = 'test'
c = a_2 + 'toast' # definitely str
return typeof(a_2), typeof(b), typeof(c)
And this should work better because it allows to infer a_1 as a double
and a_2 as a string.
>>>
>>> This already works, right?
>>
>> It would work if it was implemented. *wink*
>
> Well, we don't infer str
Yes we do, there are even some optimisations for str. It's well defined for
both Py2 and Py3, just not the same on both, so the final code to use for
them is C compile time dependent.
I meant to say that entry splitting isn't implemented.
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] CF based type inference
On Fri, May 11, 2012 at 8:56 AM, Stefan Behnel wrote:
> Robert Bradshaw, 11.05.2012 17:48:
>> On Tue, May 8, 2012 at 11:22 PM, Stefan Behnel wrote:
>>> Robert Bradshaw, 09.05.2012 00:12:
On Tue, May 8, 2012 at 6:47 AM, Vitja Makarov wrote:
> 2012/5/8 Stefan Behnel:
>> Vitja has rebased the type inference on the control flow, so I wonder if
>> this will enable us to properly infer this:
>>
>> def partial_validity():
>> """
>> >>> partial_validity()
>> ('Python object', 'double', 'str object')
>> """
>> a = 1.0
>> b = a + 2 # definitely double
>> a = 'test'
>> c = a + 'toast' # definitely str
>> return typeof(a), typeof(b), typeof(c)
>>
>> I think, what is mainly needed for this is that a NameNode with an
>> undeclared type should not report its own entry as dependency but that of
>> its own cf_assignments. Would this work?
>>
>> (Haven't got the time to try it out right now, so I'm dumping it here.)
>>
>
> Yeah, that might work. The other way to go is to split entries:
>
> def partial_validity():
> """
> >>> partial_validity()
> ('str object', 'double', 'str object')
> """
> a_1 = 1.0
> b = a_1 + 2 # definitely double
> a_2 = 'test'
> c = a_2 + 'toast' # definitely str
> return typeof(a_2), typeof(b), typeof(c)
>
> And this should work better because it allows to infer a_1 as a double
> and a_2 as a string.
This already works, right?
>>>
>>> It would work if it was implemented. *wink*
>>
>> Well, we don't infer str
>
> Yes we do, there are even some optimisations for str. It's well defined for
> both Py2 and Py3, just not the same on both, so the final code to use for
> them is C compile time dependent.
>
> I meant to say that entry splitting isn't implemented.
Yeah, that isn't implemented yet.
- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
