[Cython] Bug - PyPy tuple leak in __Pyx_PyObject_CallOneArg?
Hello,
I'm one of the maintainers of the gevent concurrency library, which has
recently been ported to run on PyPy. Under PyPy, a small portion of the code is
compiled with Cython in order to get the desired atomic semantics
(specifically, a semaphore class). We recently had a user report an easily
reproducible leak of tuples of one element. Tracking it down, it appears that
__Pyx_PyObject_CallOneArg creates a new tuple under PyPy, but neglects to free
it. This was tested with Cython 0.23.3 and PyPy 2.6.1.
Our Cython code contained a loop like this, and every iteration of the loop
leaked a tuple:
for link in links:
link(self)
The C output for that last line looked like this:
__Pyx_PyObject_CallOneArg(__pyx_t_10, ((PyObject *)__pyx_v_self));...
And, under PyPy, the implementation of __Pyx_PyObject_CallOneArg is different
than it is under CPython. Specifically, with Cython 0.23.3 this is the code:
#if CYTHON_COMPILING_IN_CPYTHON
...
#else
static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func,
PyObject *arg) {
PyObject* args = PyTuple_Pack(1, arg);
return (likely(args)) ? __Pyx_PyObject_Call(func, args, NULL) : NULL;
}
#endif
PyTuple_Pack is documented as returning a new reference
(https://docs.python.org/2/c-api/tuple.html#c.PyTuple_Pack), so it seems to me
like this code should be decrementing the reference to the tuple before
returning (much like the __Pyx__PyObject_CallOneArg function does under
CPython).
Changing the loop to avoid the use of CallOneArg seemed to resolve the tuple
leak, presumably at some execution time cost:
args = (self,)
for link in links:
link(*self)
There's some additional background at
https://bitbucket.org/pypy/pypy/issues/2149/memory-leak-for-python-subclass-of-cpyext#comment-22347393
Am I interpreting this correctly to be a bug, or could there be something wrong
in the way we're handling callbacks? Please let me know if there's any further
information I can provide.
Thanks,
Jason
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython alpha 0.24
gevent compiles and passes all its tests with this release. However, compilation produces 420 warnings of the form: warning: gevent/corecext.pyx:1893:4: 'property rpid:' syntax is deprecated, use '@property' That's a lot of warnings. As I understand it, this was added with pull #462 which is only in master/0.24, so there On Fri, Mar 11, 2016 at 2:23 AM, Robert Bradshaw wrote: > We are getting ready to push a new major release. A first alpha is up > at http://cython.org/release/Cython-0.24a0.tar.gz > > To avoid surprises, please test and report. > > Thanks, > Robert > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython alpha 0.24
[Ack, really sorry for the premature send. Here's the whole thing:] gevent compiles and passes all its tests with this release. However, compilation produces 420 warnings of the form: warning: gevent/corecext.pyx:1893:4: 'property rpid:' syntax is deprecated, use '@property' That's a lot of warnings. There's also this comment in the code, which seems to indicate that the warnings aren't actually desired, but the test for it includes `if True:` so warnings are always emitted. I'm a bit confused as to the intent; are the warnings going to be in the final release? def visit_PropertyNode(self, node): # Suppress warning for our code until we can convert all our uses over. if isinstance(node.pos[0], str) or True: warning(node.pos, "'property %s:' syntax is deprecated, use '@property'" % node.name, 2) As I understand it, this was added with pull #462 which is only in master/0.24, so there's no way for me to fix these warnings while still being able to compile on 0.23; is that right? (That is, @property doesn't work right in 0.23) If so, this would seem to at least merit a mention in CHANGES.rst? Thanks, jason On Sat, Mar 19, 2016 at 9:44 AM, Jason Madden wrote: > gevent compiles and passes all its tests with this release. > > However, compilation produces 420 warnings of the form: > > warning: gevent/corecext.pyx:1893:4: 'property rpid:' syntax is > deprecated, use '@property' > > That's a lot of warnings. > > As I understand it, this was added with pull #462 which is only in > master/0.24, so there > > > On Fri, Mar 11, 2016 at 2:23 AM, Robert Bradshaw > wrote: > >> We are getting ready to push a new major release. A first alpha is up >> at http://cython.org/release/Cython-0.24a0.tar.gz >> >> To avoid surprises, please test and report. >> >> Thanks, >> Robert >> ___ >> cython-devel mailing list >> cython-devel@python.org >> https://mail.python.org/mailman/listinfo/cython-devel >> > > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython 0.23.5 released
> On Mar 31, 2016, at 00:50, Matthew Brett wrote: > > On Wed, Mar 30, 2016 at 2:35 PM, Forest Gregg wrote: >> Thanks! >> >> Are there also plans to post Windows wheels for cython 0.23.5 to pypi like >> there was for 0.23.4? > > I did that earlier today, using an Appveyor build system : > > https://github.com/MacPython/cython-wheels/blob/master/appveyor.yml > https://ci.appveyor.com/project/matthew-brett/cython-wheels > http://win-wheels.scikit-image.org/ > > Do they work for you? They work for building gevent on Appveyor. The cython binary wheels cut gevent's total build time by more than half (they save almost 10 minutes). Thanks! Jason ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython 0.23.5 released
On Thursday, 31 March 2016 00:50:57 UTC-5, Matthew Brett wrote: > > On Wed, Mar 30, 2016 at 2:35 PM, Forest Gregg > wrote: > > Thanks! > > > > Are there also plans to post Windows wheels for cython 0.23.5 to pypi > like > > there was for 0.23.4? > > I did that earlier today, using an Appveyor build system : > > https://github.com/MacPython/cython-wheels/blob/master/appveyor.yml > https://ci.appveyor.com/project/matthew-brett/cython-wheels > http://win-wheels.scikit-image.org/ > > Do they work for you? > They work for building gevent on Appveyor. The cython binary wheels cut gevent's total build time by more than half (they save almost 10 minutes). Thanks! Jason ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] bug report on cython-mode.el: freezes when using which-function-mode
> On Apr 24, 2016, at 03:15, Nathaniel Smith wrote: > > I don't know whether the same thing happens with released versions of > emacs. I see the same behaviour with the 24.5 release of emacs (stock python.el + elpy). Turning off which-function-mode seems to solve it (thanks for the tip, BTW). ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython 0.27 beta 1 is available
> On Sep 19, 2017, at 02:49, Stefan Behnel wrote: > > Right, I forgot that alpha releases always scream for not being touched. ;) > > So here's the same thing as beta-1 to get things rolling. > > https://github.com/cython/cython/archive/0.27b1.tar.gz > > Changelog: > https://github.com/cython/cython/blob/2c9641a749208dbc405209e367b8a23e1310564d/CHANGES.rst > > Feedback is appreciated. gevent doesn't use any of the new features, but I can confirm that it does build and pass its tests under Python 2.7/3.4/3.6 with Cython 0.27b1. Jason ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
