Re: [Cython] Cython 0.23 released
Hi, I've just hit a regression with 0.23: a recursion loop generating code for this particular pyx: http://xpra.org/trac/browser/xpra/trunk/src/xpra/codecs/nvenc4/encoder.pyx I don't think we have such loops in our setup.py data structures or in that pyx, but that's possible. (I will try to check later) The same code builds fine on all platforms with Cython 0.22.1. Full stack trace attached to this email. Loop excerpt: File "Cython/Compiler/FlowControl.py", line 336, in Cython.Compiler.FlowControl.NameAssignment.__deepcopy__ (/usr/src/rpmbuild/BUILD/Cython-0.23/Cython/Compiler/FlowControl.c:10989) File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib64/python2.7/copy.py", line 328, in _reconstruct args = deepcopy(args, memo) File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib64/python2.7/copy.py", line 237, in _deepcopy_tuple y.append(deepcopy(a, memo)) File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib64/python2.7/copy.py", line 230, in _deepcopy_list y.append(deepcopy(a, memo)) File "/usr/lib64/python2.7/copy.py", line 174, in deepcopy y = copier(memo) File "Cython/Compiler/FlowControl.py", line 418, in Cython.Compiler.FlowControl.NameReference.__deepcopy__ (/usr/src/rpmbuild/BUILD/Cython-0.23/Cython/Compiler/FlowControl.c:13685) File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib64/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib64/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib64/python2.7/copy.py", line 351, in _reconstruct item = deepcopy(item, memo) File "/usr/lib64/python2.7/copy.py", line 174, in deepcopy y = copier(memo) File "Cython/Compiler/FlowControl.py", line 336, in Cython.Compiler.FlowControl.NameAssignment.__deepcopy__ (/usr/src/rpmbuild/BUILD/Cython-0.23/Cython/Compiler/FlowControl.c:10989) Sorry for not testing the 0.23 beta, which might have caught this sooner. Cheers Antoine On 09/08/15 01:47, Stefan Behnel wrote: > Hi everyone, > > given the positive feedback on the previous beta releases, I'm happy to > announce the final release of Cython 0.23. There were only minor changes > and fixes since the second beta. > > https://pypi.python.org/pypi/Cython/0.23 > > This is a major feature release. Highlights include, in addition to several > important bug fixes: > > * PEP 492 (async/await) and PEP 448 (star-unpacking generalizations) > * inlined generator expressions > * support for the C++ assignment and bool() operators > * support for coverage analysis > > Complete changelog follows below. Upgrading is generally recommended. > > > You can get the signed release from here: > > http://cython.org/ > > http://cython.org/release/Cython-0.23.tar.gz > > http://cython.org/release/Cython-0.23.zip > > > SHA1 sums: > 5a49123b39d89a20fb88bdc7886210c39cdff695 Cython-0.23.tar.gz > 9bec6c6df861be49ed608a0ab688b67bba543abe Cython-0.23.zip > > Have fun, > > Stefan > > > > 0.23 (2015-08-08) > = > > Features added > -- > > * PEP 492 (async/await) was implemented. > See https://www.python.org/dev/peps/pep-0492/ > > * PEP 448 (Additional Unpacking Generalizations) was implemented. > See https://www.python.org/dev/peps/pep-0448/ > > * Support for coverage.py 4.0+ can be enabled by adding the plugin > "Cython.Coverage" to the ".coveragerc" config file. > > * Annotated HTML source pages can integrate (XML) coverage reports. > > * Tracing is supported in ``nogil`` functions/sections and module init > code. > > * When generators are used in a Cython module and the module imports the > modules "inspect" and/or "asyncio", Cython enables interoperability by > patching these modules during the import to recognise Cython's internal > generator and coroutine types. This can be disabled by C compiling the > module with "-D CYTHON_PATCH_ASYNCIO=0" or "-D CYTHON_PATCH_INSPECT=0" > > * When generators or coroutines are used in a Cython module, their types > are registered with the ``Generator`` and ``Coroutine`` ABCs in the > ``collections`` or ``collections.abc`` stdlib module at import time to > enable interoperability with code that needs to detect and process Python > generators/coroutines. These ABCs were added in CPython 3.5 and are > available for older Python versions through the ``backports_abc`` module > on PyPI. See https://bugs.python.org/issue24018 > > * Adding/subtracting/dividing/modulus and equality comparisons with > constant Python floats and small integers are faster. > > * Bin
Re: [Cython] Cython 0.23 released
On 09/08/15 13:54, Antoine Martin wrote:
> Hi,
>
> I've just hit a regression with 0.23: a recursion loop generating code
> for this particular pyx:
> http://xpra.org/trac/browser/xpra/trunk/src/xpra/codecs/nvenc4/encoder.pyx
>
> I don't think we have such loops in our setup.py data structures or in
> that pyx, but that's possible. (I will try to check later)
> The same code builds fine on all platforms with Cython 0.22.1.
I have narrowed it down to the problematic finally block via trial and
error:
http://xpra.org/trac/browser/xpra/trunk/src/xpra/codecs/nvenc4/encoder.pyx#L2024
Any of those statements, including the simple logging ones, cause
Cython's "fresh_finally_clause" code to go into an infinite loop via
deepcopy.
This "fixes" it by moving the statements out of the finally block
(obviously not a practical solution):
--- xpra/codecs/nvenc4/encoder.pyx(revision 10311)
+++ xpra/codecs/nvenc4/encoder.pyx(working copy)
@@ -2022,16 +2022,17 @@
self.bytes_out += size
data = ( lockOutputBuffer.bitstreamBufferPtr)[:size]
finally:
-if DEBUG_API:
-log("nvEncUnlockBitstream(%#x)",
self.bitstreamBuffer)
-with nogil:
-r =
self.functionList.nvEncUnlockBitstream(self.context, self.bitstreamBuffer)
-raiseNVENC(r, "unlocking output buffer")
-if DEBUG_API:
-log("nvEncUnmapInputResource(%#x)",
self.bitstreamBuffer)
-with nogil:
-r =
self.functionList.nvEncUnmapInputResource(self.context,
mapInputResource.mappedResource)
-raiseNVENC(r, "unmapping input resource")
+pass
+if DEBUG_API:
+log("nvEncUnlockBitstream(%#x)",
self.bitstreamBuffer)
+with nogil:
+r = self.functionList.nvEncUnlockBitstream(self.context,
self.bitstreamBuffer)
+raiseNVENC(r, "unlocking output buffer")
+if DEBUG_API:
+log("nvEncUnmapInputResource(%#x)",
self.bitstreamBuffer)
+with nogil:
+r = self.functionList.nvEncUnmapInputResource(self.context,
mapInputResource.mappedResource)
+raiseNVENC(r, "unmapping input resource")
download_end = time.time()
log("compress_image(..) download took %.1f ms",
(download_end-encode_end)*1000.0)
What am I doing wrong here?
Those statements look fine and they used to compile just fine too.
Thanks
Antoine
>
> Full stack trace attached to this email. Loop excerpt:
> File "Cython/Compiler/FlowControl.py", line 336, in
> Cython.Compiler.FlowControl.NameAssignment.__deepcopy__
> (/usr/src/rpmbuild/BUILD/Cython-0.23/Cython/Compiler/FlowControl.c:10989)
> File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy
> y = _reconstruct(x, rv, 1, memo)
> File "/usr/lib64/python2.7/copy.py", line 328, in _reconstruct
> args = deepcopy(args, memo)
> File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy
> y = copier(x, memo)
> File "/usr/lib64/python2.7/copy.py", line 237, in _deepcopy_tuple
> y.append(deepcopy(a, memo))
> File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy
> y = copier(x, memo)
> File "/usr/lib64/python2.7/copy.py", line 230, in _deepcopy_list
> y.append(deepcopy(a, memo))
> File "/usr/lib64/python2.7/copy.py", line 174, in deepcopy
> y = copier(memo)
> File "Cython/Compiler/FlowControl.py", line 418, in
> Cython.Compiler.FlowControl.NameReference.__deepcopy__
> (/usr/src/rpmbuild/BUILD/Cython-0.23/Cython/Compiler/FlowControl.c:13685)
> File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy
> y = _reconstruct(x, rv, 1, memo)
> File "/usr/lib64/python2.7/copy.py", line 334, in _reconstruct
> state = deepcopy(state, memo)
> File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy
> y = copier(x, memo)
> File "/usr/lib64/python2.7/copy.py", line 257, in _deepcopy_dict
> y[deepcopy(key, memo)] = deepcopy(value, memo)
> File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy
> y = _reconstruct(x, rv, 1, memo)
> File "/usr/lib64/python2.7/copy.py", line 351, in _reconstruct
> item = deepcopy(item, memo)
> File "/usr/lib64/python2.7/copy.py", line 174, in deepcopy
> y = copier(memo)
> File "Cython/Compiler/FlowControl.py", line 336, in
> Cython.Compiler.FlowControl.NameAssignment.__deepcopy__
> (/usr/src/rpmbuild/BUILD/Cython-0.23/Cython/Compiler/FlowControl.c:10989)
>
> Sorry for not testing the 0.23 beta, which might have caught this sooner.
>
> C
Re: [Cython] Cython 0.23 released
On 17/08/15 14:14, Stefan Behnel wrote: > Stefan Behnel schrieb am 17.08.2015 um 08:17: >> Antoine Martin schrieb am 17.08.2015 um 06:09: >>> On 09/08/15 13:54, Antoine Martin wrote: >>>> I've just hit a regression with 0.23: a recursion loop generating code >>>> for this particular pyx: >>>> http://xpra.org/trac/browser/xpra/trunk/src/xpra/codecs/nvenc4/encoder.pyx >>> I have narrowed it down to the problematic finally block via trial and >>> error: >>> http://xpra.org/trac/browser/xpra/trunk/src/xpra/codecs/nvenc4/encoder.pyx#L2024 >>> Any of those statements, including the simple logging ones, cause >>> Cython's "fresh_finally_clause" code to go into an infinite loop via >>> deepcopy. >>> >>> What am I doing wrong here? >>> Those statements look fine and they used to compile just fine too. >> Thanks for the report. This is a bug in the latest Cython release. I'll see >> if I can come up with a fix. > I couldn't reproduce it with any reasonable amount of test code yet, but > does this fix it for you? > > https://github.com/cython/cython/commit/5a40d5459aad1c6f15fb8526f0144d3c0f1dfe76 I am afraid not, I still get the same stacktrace. Let me know if there's anything else I can try. Cheers Antoine > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.23 released
On 19/08/15 01:03, Stefan Behnel wrote: > Antoine Martin schrieb am 17.08.2015 um 14:14: >> On 17/08/15 14:14, Stefan Behnel wrote: >>> Stefan Behnel schrieb am 17.08.2015 um 08:17: >>>> Antoine Martin schrieb am 17.08.2015 um 06:09: >>>>> On 09/08/15 13:54, Antoine Martin wrote: >>>>>> I've just hit a regression with 0.23: a recursion loop generating code >>>>>> for this particular pyx: >>>>>> http://xpra.org/trac/browser/xpra/trunk/src/xpra/codecs/nvenc4/encoder.pyx >>>>> I have narrowed it down to the problematic finally block via trial and >>>>> error: >>>>> http://xpra.org/trac/browser/xpra/trunk/src/xpra/codecs/nvenc4/encoder.pyx#L2024 >>>>> Any of those statements, including the simple logging ones, cause >>>>> Cython's "fresh_finally_clause" code to go into an infinite loop via >>>>> deepcopy. >>>>> >>>>> What am I doing wrong here? >>>>> Those statements look fine and they used to compile just fine too. >>>> Thanks for the report. This is a bug in the latest Cython release. I'll see >>>> if I can come up with a fix. >>> I couldn't reproduce it with any reasonable amount of test code yet, but >>> does this fix it for you? >>> >>> https://github.com/cython/cython/commit/5a40d5459aad1c6f15fb8526f0144d3c0f1dfe76 >> I am afraid not, I still get the same stacktrace. >> Let me know if there's anything else I can try. > How about this? > > https://github.com/cython/cython/commit/5fa766ac25a31b0c1addd08efd19d2d34661317b That fixes it! Thank you very much. Antoine > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.25 released
On 26/10/16 03:34, Robert Bradshaw wrote: > I'm happy to announce the release of Cython 0.25 which has numerous > new features and bug fixes. It is available at > https://pypi.python.org/pypi/Cython > > We have also moved bug tracking from trac.cython.org to github issues. > However, user support still remains at cython-users@, please refrain > from filing issues until you have confirmed an actual bug. 0.25 does not build on any CentOS 6.x: wget "https://pypi.python.org/packages/f8/2e/5898046f8089205981447c23ebb8fe02cd9d66939cd74338aa4872853d8e/Cython-0.25.tar.gz#md5=1b61433b8410ac382ac9b248b42466fd"; tar -zxf Cython-0.25.tar.gz cd Cython-0.25 python ./setup.py build (..) building 'Cython.Runtime.refnanny' extension creating build/temp.linux-x86_64-2.6/home/centos/Cython-0.25/Cython/Runtime gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c /home/centos/Cython-0.25/Cython/Runtime/refnanny.c -o build/temp.linux-x86_64-2.6/home/centos/Cython-0.25/Cython/Runtime/refnanny.o gcc: /home/centos/Cython-0.25/Cython/Runtime/refnanny.c: No such file or directory gcc: no input files error: command 'gcc' failed with exit status 1 If I just run: cythonize /home/centos/Cython-0.25/Cython/Runtime/refnanny.pyx Then the build succeeds. Cheers Antoine > > - Robert > > > Features added > -- > > * def/cpdef methods of cdef classes benefit from Cython's internal function > implementation, which enables introspection and line profiling for them. > Implementation sponsored by Turbostream (www.turbostream-cfd.com). > > * The distutils extension ``Cython.Distutils.build_ext`` has now been updated > to use cythonize which properly handles dependencies. The old extension can > still be found in ``Cython.Distutils.old_build_ext`` and is now deprecated. > > * Calls to Python functions are faster, following the recent "FastCall" > optimisations that Victor Stinner implemented for CPython 3.6. > See https://bugs.python.org/issue27128 and related issues. > > * The new METH_FASTCALL calling convention for PyCFunctions is supported > in CPython 3.6. See https://bugs.python.org/issue27810 > > * C++ classes can now have typedef members. STL containers updated with > value_type. > > * Support for bazel using a the pyx_library rule in //Tools:rules.bzl. > > * Initial support for using Cython modules in Pyston. Patch by Daetalus. > > * Dynamic Python attributes are allowed on cdef classes if an attribute > ``cdef dict __dict__`` is declared in the class. Patch by empyrical. > > * Cython implemented C++ classes can make direct calls to base class methods. > Patch by empyrical. > > * New directive ``cython.no_gc`` to fully disable GC for a cdef class. > Patch by Claudio Freire. > > * Buffer variables are no longer excluded from ``locals()``. > Patch by da-woods. > > * Building f-strings is faster, especially when formatting C integers. > > * for-loop iteration over "std::string". > > * ``libc/math.pxd`` provides ``e`` and ``pi`` as alias constants to simplify > usage as a drop-in replacement for Python's math module. > > * Speed up cython.inline(). > > * Binary lshift operations with small constant Python integers are faster. > > * Some integer operations on Python long objects are faster in Python 2.7. > > * Support for the C++ ``typeid`` operator. > > Significant Bugs fixed > -- > > * Division of complex numbers avoids overflow by using Smith's method. > > * Some function signatures in ``libc.math`` and ``numpy.pxd`` were incorrect. > Patch by Michael Seifert. > > Other changes > - > > * The "%%cython" IPython/jupyter magic now defaults to the language level of > the current jupyter kernel. The language level can be set explicitly with > "%%cython -2" or "%%cython -3". > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
