[Cython] Upcoming cython/numpy breakage with stride checking
Hi all, If you build current numpy master with NPY_RELAXED_STRIDES_CHECKING=1 python setup.py install then Cython code using ndarrays starts blowing up, e.g.: # foo.pyx def add_one(array): cdef double[::1] a = array a[0] += 1. return array >>> foo.add_one(np.ascontiguousarray(np.arange(10.)[::100])) Traceback (most recent call last): File "", line 1, in File "foo.pyx", line 2, in foo.add_one (foo.c:1210) cdef double[::1] a = array ValueError: Buffer and memoryview are not contiguous in the same dimension. The problem (as discussed before) is that Cython has an unnecessarily strict definition of "contiguous", so NPY_RELAXED_STRIDES_CHECKING=1 pretty much breaks all existing compiled Cython modules. Our plan is to make NPY_RELAXED_STRIDES_CHECKING=1 into the default sooner or later, and Cython is a major blocker on this plan. It may become the default as soon as the 1.8 pre-releases (with the expectation that we'll probably have to switch back again before the actual release, but still). References: Previous thread: http://thread.gmane.org/gmane.comp.python.cython.devel/14634 Detailed discussion of the difference between numpy/cython's current definition of "contiguity", and the correct definition: http://thread.gmane.org/gmane.comp.python.cython.devel/14634/focus=14640 The PR implementing NPY_RELAXED_STRIDES_CHECKING: https://github.com/numpy/numpy/pull/3162 Another test case: https://github.com/numpy/numpy/issues/2956 We're hoping that Cython will also switch soon to the more accurate check for contiguity. This shouldn't cause any backwards compatibility problems -- it just means Cython code would make strictly fewer copies, and error out due to lack of contiguity strictly less often, even with older numpys. And it seems like a necessary step for getting this untangled and minimizing user pain. What do you think? -n ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Fwd: Re: JIT compilers for Python, what is the latest news?
Maybe we should update the docs here? Original-Message Subject: Re: JIT compilers for Python, what is the latest news? Date: Sat, 6 Apr 2013 11:23:43 +0100 From: Joshua Landau To: gmane.comp.python.general On 5 April 2013 19:37, Devin Jeanpierre wrote: > On Fri, Apr 5, 2013 at 4:34 AM, John Ladasky wrote: >> On Thursday, April 4, 2013 7:39:16 PM UTC-7, MRAB wrote: >>> Have you looked at Cython? Not quite the same, but still... >> >> I'm already using Numpy, compiled with what is supposed to be a fast >> LAPACK. I don't think I want to attempt to improve on all the work that >> has gone into Numpy. > > There's no reason you can't use both cython and numpy. See: > http://docs.cython.org/src/tutorial/numpy.html Don't use this. Use memoryviews: http://docs.cython.org/src/userguide/memoryviews.html. I have no idea why that doc page isn't headed "DEPRICATED" by now. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython classes and attribute cache
Nils Bruin, 06.04.2013 21:37: > I can confirm that changing Compiler/TypeSlots.py, line 380 from: > > value = "Py_TPFLAGS_DEFAULT|Py_TPFLAGS_CHECKTYPES| > Py_TPFLAGS_HAVE_NEWBUFFER" > > to: > > value = "Py_TPFLAGS_DEFAULT|Py_TPFLAGS_CHECKTYPES| > Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_HAVE_VERSION_TAG" > > indeed makes the difference in timing disappear. For extension classes > that don't mess directly with their tp_dict having this flag should be > entirely safe. > > See http://bugs.python.org/issue1700288 for background on the > introduction of the method cache in python and > http://bugs.python.org/issue1878 > for the reason why the flag is not included in Py_TPFLAGS_DEFAULT on > 2.7, but is in 3.*. I wouldn't mind making it an "on by default" compiler directive. That would mean that you could switch it off with a class decorator at need. Any objections? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython classes and attribute cache
Stefan Behnel, 06.04.2013 21:58: > Nils Bruin, 06.04.2013 21:37: >> I can confirm that changing Compiler/TypeSlots.py, line 380 from: >> >> value = "Py_TPFLAGS_DEFAULT|Py_TPFLAGS_CHECKTYPES| >> Py_TPFLAGS_HAVE_NEWBUFFER" >> >> to: >> >> value = "Py_TPFLAGS_DEFAULT|Py_TPFLAGS_CHECKTYPES| >> Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_HAVE_VERSION_TAG" >> >> indeed makes the difference in timing disappear. For extension classes >> that don't mess directly with their tp_dict having this flag should be >> entirely safe. >> >> See http://bugs.python.org/issue1700288 for background on the >> introduction of the method cache in python and >> http://bugs.python.org/issue1878 >> for the reason why the flag is not included in Py_TPFLAGS_DEFAULT on >> 2.7, but is in 3.*. > > I wouldn't mind making it an "on by default" compiler directive. That would > mean that you could switch it off with a class decorator at need. Proposed patch: https://github.com/scoder/cython/commit/b1f10fa2c1bbb0a076807a9f391204c6a997ba9a > Any objections? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
