Re: [Cython] inspect.isbuiltin(cyfunction) and inheritance from PyCFunction
Stefan Behnel, 01.02.2014 17:04: > Stefan Behnel, 31.01.2014 21:01: >> Yury Selivanov just committed a revised version of a patch I wrote for >> CPython's inspect module last year. It now accepts Cython's function type >> as Python function, based on the function-like attributes that it exports. >> >> http://hg.python.org/cpython/rev/32a660a52aae >> >> That means that things like inspect.signature(cyfunction) will also work >> now, and finally have a reason to be further improved, including parameter >> introspection, annotations, etc. :) > > Sorry, huge correction: inspect.isfunction(cyfunction) still returns False. > Only introspection and signature analysis in the inspect module were > changed. I got it wrong because I had code in my usersitecustomize.py that > monkey patches inspect.isfunction(), and had completely forgotten about it. > > Looking through this a bit more, I noticed that CyFunction inherits from > PyCFunction, but doesn't actually tell CPython that it does. If I add this > line to __Pyx_CyFunction_init(), before the PyType_Ready() call: > > __pyx_CyFunctionType_type.tp_base = &PyCFunction_Type; > > it still works as expected, but inspect.isbuiltin() now returns True (as > expected). Does anyone see a reason why we should not be doing this? Tried it, found a reason. It makes CPython inherit some slot functions from the base type, specifically tp_hash and tp_richompare, and those don't work with the cyfunction type. Meaning, we'd have to implement those explicitly to override the base type's implementations. Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] inspect.isbuiltin(cyfunction) and inheritance from PyCFunction
Stefan Behnel, 02.02.2014 11:07: > Stefan Behnel, 01.02.2014 17:04: >> Stefan Behnel, 31.01.2014 21:01: >>> Yury Selivanov just committed a revised version of a patch I wrote for >>> CPython's inspect module last year. It now accepts Cython's function type >>> as Python function, based on the function-like attributes that it exports. >>> >>> http://hg.python.org/cpython/rev/32a660a52aae >>> >>> That means that things like inspect.signature(cyfunction) will also work >>> now, and finally have a reason to be further improved, including parameter >>> introspection, annotations, etc. :) >> >> Sorry, huge correction: inspect.isfunction(cyfunction) still returns False. >> Only introspection and signature analysis in the inspect module were >> changed. I got it wrong because I had code in my usersitecustomize.py that >> monkey patches inspect.isfunction(), and had completely forgotten about it. >> >> Looking through this a bit more, I noticed that CyFunction inherits from >> PyCFunction, but doesn't actually tell CPython that it does. If I add this >> line to __Pyx_CyFunction_init(), before the PyType_Ready() call: >> >> __pyx_CyFunctionType_type.tp_base = &PyCFunction_Type; >> >> it still works as expected, but inspect.isbuiltin() now returns True (as >> expected). Does anyone see a reason why we should not be doing this? > > Tried it, found a reason. It makes CPython inherit some slot functions from > the base type, specifically tp_hash and tp_richompare, and those don't work > with the cyfunction type. Meaning, we'd have to implement those explicitly > to override the base type's implementations. I think I figured it out. The fact that the object struct starts with "PyCFunctionObject" is more of a convenience. There is no "real" inheritance relation between the two, and that's the right thing to do, also. So, the situation in Py3.4 is as follows. inspect.isfunction(cyfunction) -> False inspect.isbuiltin(cyfunction) -> False inspect.isroutine(cyfunction) -> True inspect.signature(cyfunction) -> Signature The fact that isroutine() returns True seems like a coincidence, based on the fact that ismethoddescriptor() returns True, which sounds kind of wrong. It just tests for an existing "__get__" on the type without a matching "__set__". Anyway, I think that's pretty much as good as we can ask for, given that Cython's function type really just quacks like a Python function. Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bug: _PyType_Lookup shortcut in Cython 0.20 breaks lookup of __exit__ in Python 2.6
[forwarding to cython-devel]
Wouter Bolsterlee, 02.02.2014 16:40:
> Cython 0.20 introduced __Pyx_PyObject_LookupSpecial as a shortcut to
> lookup special methods. The commit that introduced this change is
> 8f6412275c4c2d1dcf43ad40306f858b114104ed and can be seen here:
>
> https://github.com/cython/cython/commit/8f6412275c4c2d1dcf43ad40306f858b114104ed
>
> It seems this change does not work with Python 2.6. The newly introduced
> function calls _PyType_Lookup(), which fails to lookup (at least)
> __exit__ (on a threading.Lock) instance when run under Python 2.6. The
> exact call is here:
>
> https://github.com/cython/cython/commit/8f6412275c4c2d1dcf43ad40306f858b114104ed#diff-83083bdce5dcca284394ead83b8d3f99R1053
>
> For my Plyvel project (https://plyvel.readthedocs.org/ and
> https://github.com/wbolster/plyvel) this issue manifests as follows:
>
> $ cython --version
> Cython version 0.20
>
> $ python
> Python 2.6.8 (unknown, Jan 26 2013, 14:35:25)
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more
> information.
> >>> import plyvel
> >>> db = plyvel.DB('/tmp/testdb', create_if_missing=True)
> >>> db.close()
> Traceback (most recent call last):
> File "", line 1, in
> File "_plyvel.pyx", line 246, in plyvel._plyvel.DB.close
> (plyvel/_plyvel.cpp:3563)
> File "_plyvel.pyx", line 252, in plyvel._plyvel.DB.close
> (plyvel/_plyvel.cpp:3221)
> AttributeError: __exit__
> >>>
> Exception AttributeError: '__exit__' in
> 'plyvel._plyvel.DB.__dealloc__' ignored
>
> The same Cython source code works fine against Python 2.7, 3.2, and 3.3.
> Reverting to Cython 0.19 makes the code run correctly on Python 2.6,
> 2.7, 3.2, and 3.3 again. This means only the combination of Python 2.6
> and Cython 0.20 fails.
>
> To check for yourself, you can run "make test" on a Git clone of Plyvel
> with either Cython 0.19 or Cython 0.20 installed, and spot the
> differences. Or run "tox" to automatically test against multiple Python
> versions.
Ok, I don't know what the exact difference is, but I do believe you that
Py2.6 doesn't play as nicely here as Py3. In any case, the CPython code
that I adapted it from didn't actually exist in Py2.6/3.1, so the
guarantees don't need to be there either.
Given that this is essentially an optimisation (and otherwise meant to
*improve* the compatibility with CPython's own behaviour), I'll disable
this in Py2.6 and Py3.1 for the next bug fix release.
https://github.com/cython/cython/commit/b8e37bc15373f303691de2d256ad99045221c9e0
It would be good if you could come up with a failing regression test case
for this, so that we can make sure it actually keeps working in the future.
> Please CC me when responding, because I'm not subscribed to
> cython-devel.
cython-devel is actually fairly low traffic, so temporarily subscribing to
it won't hurt much.
Stefan
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bug: _PyType_Lookup shortcut in Cython 0.20 breaks lookup of __exit__ in Python 2.6
Hi, Thanks for the quick follow-up. See inline comments below. Stefan Behnel schreef op zo 02-02-2014 om 18:13 [+0100]: > Ok, I don't know what the exact difference is, but I do believe you > that > Py2.6 doesn't play as nicely here as Py3. In any case, the CPython > code > that I adapted it from didn't actually exist in Py2.6/3.1, so the > guarantees don't need to be there either. I didn't really look into this, so you may be right here. > Given that this is essentially an optimisation (and otherwise meant to > *improve* the compatibility with CPython's own behaviour), I'll > disable > this in Py2.6 and Py3.1 for the next bug fix release. > > https://github.com/cython/cython/commit/b8e37bc15373f303691de2d256ad99045221c9e0 > > It would be good if you could come up with a failing regression test > case > for this, so that we can make sure it actually keeps working in the > future. I'm not familiar with the Cython codebase at all, so this might be a bit too hard for me. However, I did manage to produce a minimal example that triggers the issue at hand. I've put up the relevant files here: https://gist.github.com/wbolster/8771899 Hopefully you'll be able to construct a test from this. — Wouter signature.asc Description: This is a digitally signed message part ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bug: _PyType_Lookup shortcut in Cython 0.20 breaks lookup of __exit__ in Python 2.6
Wouter Bolsterlee, 02.02.2014 18:43: > https://gist.github.com/wbolster/8771899 > > Hopefully you'll be able to construct a test from this. Thanks! https://github.com/cython/cython/commit/e40d032f4a1e851c9bc897c36176483b6c92cce8 Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
