Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
mark florisson, 03.01.2014 23:28: > On 3 January 2014 18:22, Stefan Behnel wrote: >> I enabled the NumPy build for our Py3.3 test runs and while I was at it, I >> got it to use the latest NumPy release 1.8. This made one of the tests fail: >> >> """ >> Traceback (most recent call last): >> File ".../doctest.py", line 1313, in __run >> compileflags, 1), test.globs) >> File "> 29)[3]>", line 1, in >> test_one_sized(a)[0] >> File "relaxed_strides.pyx", line 38, in >> relaxed_strides.test_one_sized (relaxed_strides.cpp:1414) >> File "stringsource", line 622, in View.MemoryView.memoryview_cwrapper >> (relaxed_strides.cpp:7568) >> File "stringsource", line 327, in >> View.MemoryView.memoryview.__cinit__ (relaxed_strides.cpp:3717) >> >> ValueError: ndarray is not C-contiguous >> """ >> >> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/1787/ARCH=m64,BACKEND=cpp,PYVERSION=py33m/console >> >> According to the comments in the test file and the corresponding NumPy pull >> request, this seems to be somewhat expected. >> >> https://github.com/cython/cython/blob/master/tests/memoryview/relaxed_strides.pyx >> >> https://github.com/numpy/numpy/pull/3162 >> >> Does someone know enough about this to figure out what to do? > > It seems to come from the call to __Pyx_GetBuffer, which is > PyObject_GetBuffer in python 3. Maybe this is Python 3 not checking > for an extent of 1, but instead only checking the stride (which is a > multiple of the itemsize)? No, Py3 doesn't do any validation here, it does a straight call into the object's slot function, i.e. into the NumPy array itself. So, the question is: who's wrong here? The test or NumPy? Hmm, or maybe just me. I didn't define the NPY_RELAXED_STRIDES_CHECKING environment variable for the NumPy build. Let's try that first. http://docs.scipy.org/doc/numpy/release.html#npy-relaxed-strides-checking Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
Stefan Behnel, 04.01.2014 12:53: > mark florisson, 03.01.2014 23:28: >> On 3 January 2014 18:22, Stefan Behnel wrote: >>> I enabled the NumPy build for our Py3.3 test runs and while I was at it, I >>> got it to use the latest NumPy release 1.8. This made one of the tests fail: >>> >>> """ >>> Traceback (most recent call last): >>> File ".../doctest.py", line 1313, in __run >>> compileflags, 1), test.globs) >>> File ">> 29)[3]>", line 1, in >>> test_one_sized(a)[0] >>> File "relaxed_strides.pyx", line 38, in >>> relaxed_strides.test_one_sized (relaxed_strides.cpp:1414) >>> File "stringsource", line 622, in View.MemoryView.memoryview_cwrapper >>> (relaxed_strides.cpp:7568) >>> File "stringsource", line 327, in >>> View.MemoryView.memoryview.__cinit__ (relaxed_strides.cpp:3717) >>> >>> ValueError: ndarray is not C-contiguous >>> """ >>> >>> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/1787/ARCH=m64,BACKEND=cpp,PYVERSION=py33m/console >>> >>> According to the comments in the test file and the corresponding NumPy pull >>> request, this seems to be somewhat expected. >>> >>> https://github.com/cython/cython/blob/master/tests/memoryview/relaxed_strides.pyx >>> >>> https://github.com/numpy/numpy/pull/3162 >>> >>> Does someone know enough about this to figure out what to do? >> >> It seems to come from the call to __Pyx_GetBuffer, which is >> PyObject_GetBuffer in python 3. Maybe this is Python 3 not checking >> for an extent of 1, but instead only checking the stride (which is a >> multiple of the itemsize)? > > No, Py3 doesn't do any validation here, it does a straight call into the > object's slot function, i.e. into the NumPy array itself. > > So, the question is: who's wrong here? The test or NumPy? > > Hmm, or maybe just me. I didn't define the NPY_RELAXED_STRIDES_CHECKING > environment variable for the NumPy build. Let's try that first. > > http://docs.scipy.org/doc/numpy/release.html#npy-relaxed-strides-checking Yep, that was it. Seems to work nicely now. Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
Excellent, thanks. I'll make sure this gets into the release. On Sat, Jan 4, 2014 at 6:46 AM, Stefan Behnel wrote: > Stefan Behnel, 04.01.2014 12:53: >> mark florisson, 03.01.2014 23:28: >>> On 3 January 2014 18:22, Stefan Behnel wrote: I enabled the NumPy build for our Py3.3 test runs and while I was at it, I got it to use the latest NumPy release 1.8. This made one of the tests fail: """ Traceback (most recent call last): File ".../doctest.py", line 1313, in __run compileflags, 1), test.globs) File ">>> 29)[3]>", line 1, in test_one_sized(a)[0] File "relaxed_strides.pyx", line 38, in relaxed_strides.test_one_sized (relaxed_strides.cpp:1414) File "stringsource", line 622, in View.MemoryView.memoryview_cwrapper (relaxed_strides.cpp:7568) File "stringsource", line 327, in View.MemoryView.memoryview.__cinit__ (relaxed_strides.cpp:3717) ValueError: ndarray is not C-contiguous """ https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/1787/ARCH=m64,BACKEND=cpp,PYVERSION=py33m/console According to the comments in the test file and the corresponding NumPy pull request, this seems to be somewhat expected. https://github.com/cython/cython/blob/master/tests/memoryview/relaxed_strides.pyx https://github.com/numpy/numpy/pull/3162 Does someone know enough about this to figure out what to do? >>> >>> It seems to come from the call to __Pyx_GetBuffer, which is >>> PyObject_GetBuffer in python 3. Maybe this is Python 3 not checking >>> for an extent of 1, but instead only checking the stride (which is a >>> multiple of the itemsize)? >> >> No, Py3 doesn't do any validation here, it does a straight call into the >> object's slot function, i.e. into the NumPy array itself. >> >> So, the question is: who's wrong here? The test or NumPy? >> >> Hmm, or maybe just me. I didn't define the NPY_RELAXED_STRIDES_CHECKING >> environment variable for the NumPy build. Let's try that first. >> >> http://docs.scipy.org/doc/numpy/release.html#npy-relaxed-strides-checking > > Yep, that was it. Seems to work nicely now. > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
Robert Bradshaw, 04.01.2014 17:40: > On Sat, Jan 4, 2014 at 6:46 AM, Stefan Behnel wrote: >> Stefan Behnel, 04.01.2014 12:53: >>> I didn't define the NPY_RELAXED_STRIDES_CHECKING >>> environment variable for the NumPy build. Let's try that first. >>> >>> http://docs.scipy.org/doc/numpy/release.html#npy-relaxed-strides-checking >> >> Yep, that was it. Seems to work nicely now. > > Excellent, thanks. I'll make sure this gets into the release. It's not a change in Cython, just a config option for the NumPy build (i.e. I changed it in the Jenkins jobs that install the external packages into CPython). Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
On 4 Jan 2014 11:53, "Stefan Behnel" wrote: > > mark florisson, 03.01.2014 23:28: > > On 3 January 2014 18:22, Stefan Behnel wrote: > >> I enabled the NumPy build for our Py3.3 test runs and while I was at it, I > >> got it to use the latest NumPy release 1.8. This made one of the tests fail: > >> > >> """ > >> Traceback (most recent call last): > >> File ".../doctest.py", line 1313, in __run > >> compileflags, 1), test.globs) > >> File " >> 29)[3]>", line 1, in > >> test_one_sized(a)[0] > >> File "relaxed_strides.pyx", line 38, in > >> relaxed_strides.test_one_sized (relaxed_strides.cpp:1414) > >> File "stringsource", line 622, in View.MemoryView.memoryview_cwrapper > >> (relaxed_strides.cpp:7568) > >> File "stringsource", line 327, in > >> View.MemoryView.memoryview.__cinit__ (relaxed_strides.cpp:3717) > >> > >> ValueError: ndarray is not C-contiguous > >> """ > >> > >> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/1787/ARCH=m64,BACKEND=cpp,PYVERSION=py33m/console > >> > >> According to the comments in the test file and the corresponding NumPy pull > >> request, this seems to be somewhat expected. > >> > >> https://github.com/cython/cython/blob/master/tests/memoryview/relaxed_strides.pyx > >> > >> https://github.com/numpy/numpy/pull/3162 > >> > >> Does someone know enough about this to figure out what to do? > > > > It seems to come from the call to __Pyx_GetBuffer, which is > > PyObject_GetBuffer in python 3. Maybe this is Python 3 not checking > > for an extent of 1, but instead only checking the stride (which is a > > multiple of the itemsize)? > > No, Py3 doesn't do any validation here, it does a straight call into the > object's slot function, i.e. into the NumPy array itself. > > So, the question is: who's wrong here? The test or NumPy? > > Hmm, or maybe just me. I didn't define the NPY_RELAXED_STRIDES_CHECKING > environment variable for the NumPy build. Let's try that first. > > http://docs.scipy.org/doc/numpy/release.html#npy-relaxed-strides-checking It's probably nicer though to write the Cython tests in such a way that they can pass against a default numpy installation? To avoid user confusion and all that. If those tests depend on numpy having relaxed strides enabled, then I'd suggest checking for this a test time and skipping the tests if not found. The relaxed strides docs give a recipe for determining how numpy was built. -n ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
Yes, that'd be good. Do you know how? I won't have time to look at this 'till next week. On Jan 4, 2014 9:54 AM, "Nathaniel Smith" wrote: > On 4 Jan 2014 11:53, "Stefan Behnel" wrote: > > > > mark florisson, 03.01.2014 23:28: > > > On 3 January 2014 18:22, Stefan Behnel wrote: > > >> I enabled the NumPy build for our Py3.3 test runs and while I was at > it, I > > >> got it to use the latest NumPy release 1.8. This made one of the > tests fail: > > >> > > >> """ > > >> Traceback (most recent call last): > > >> File ".../doctest.py", line 1313, in __run > > >> compileflags, 1), test.globs) > > >> File " > >> 29)[3]>", line 1, in > > >> test_one_sized(a)[0] > > >> File "relaxed_strides.pyx", line 38, in > > >> relaxed_strides.test_one_sized (relaxed_strides.cpp:1414) > > >> File "stringsource", line 622, in > View.MemoryView.memoryview_cwrapper > > >> (relaxed_strides.cpp:7568) > > >> File "stringsource", line 327, in > > >> View.MemoryView.memoryview.__cinit__ (relaxed_strides.cpp:3717) > > >> > > >> ValueError: ndarray is not C-contiguous > > >> """ > > >> > > >> > https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/1787/ARCH=m64,BACKEND=cpp,PYVERSION=py33m/console > > >> > > >> According to the comments in the test file and the corresponding > NumPy pull > > >> request, this seems to be somewhat expected. > > >> > > >> > https://github.com/cython/cython/blob/master/tests/memoryview/relaxed_strides.pyx > > >> > > >> https://github.com/numpy/numpy/pull/3162 > > >> > > >> Does someone know enough about this to figure out what to do? > > > > > > It seems to come from the call to __Pyx_GetBuffer, which is > > > PyObject_GetBuffer in python 3. Maybe this is Python 3 not checking > > > for an extent of 1, but instead only checking the stride (which is a > > > multiple of the itemsize)? > > > > No, Py3 doesn't do any validation here, it does a straight call into the > > object's slot function, i.e. into the NumPy array itself. > > > > So, the question is: who's wrong here? The test or NumPy? > > > > Hmm, or maybe just me. I didn't define the NPY_RELAXED_STRIDES_CHECKING > > environment variable for the NumPy build. Let's try that first. > > > > > http://docs.scipy.org/doc/numpy/release.html#npy-relaxed-strides-checking > > It's probably nicer though to write the Cython tests in such a way that > they can pass against a default numpy installation? To avoid user confusion > and all that. > > If those tests depend on numpy having relaxed strides enabled, then I'd > suggest checking for this a test time and skipping the tests if not found. > The relaxed strides docs give a recipe for determining how numpy was built. > > -n > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
On 4 Jan 2014 17:07, "Robert Bradshaw" wrote: > > Yes, that'd be good. Do you know how? I won't have time to look at this 'till next week. Something like if not np.ones((10, 1), order="C").flags.f_contiguous: # numpy without relaxed stride support raise SkipTest > On Jan 4, 2014 9:54 AM, "Nathaniel Smith" wrote: >> >> On 4 Jan 2014 11:53, "Stefan Behnel" wrote: >> > >> > mark florisson, 03.01.2014 23:28: >> > > On 3 January 2014 18:22, Stefan Behnel wrote: >> > >> I enabled the NumPy build for our Py3.3 test runs and while I was at it, I >> > >> got it to use the latest NumPy release 1.8. This made one of the tests fail: >> > >> >> > >> """ >> > >> Traceback (most recent call last): >> > >> File ".../doctest.py", line 1313, in __run >> > >> compileflags, 1), test.globs) >> > >> File "> > >> 29)[3]>", line 1, in >> > >> test_one_sized(a)[0] >> > >> File "relaxed_strides.pyx", line 38, in >> > >> relaxed_strides.test_one_sized (relaxed_strides.cpp:1414) >> > >> File "stringsource", line 622, in View.MemoryView.memoryview_cwrapper >> > >> (relaxed_strides.cpp:7568) >> > >> File "stringsource", line 327, in >> > >> View.MemoryView.memoryview.__cinit__ (relaxed_strides.cpp:3717) >> > >> >> > >> ValueError: ndarray is not C-contiguous >> > >> """ >> > >> >> > >> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/1787/ARCH=m64,BACKEND=cpp,PYVERSION=py33m/console >> > >> >> > >> According to the comments in the test file and the corresponding NumPy pull >> > >> request, this seems to be somewhat expected. >> > >> >> > >> https://github.com/cython/cython/blob/master/tests/memoryview/relaxed_strides.pyx >> > >> >> > >> https://github.com/numpy/numpy/pull/3162 >> > >> >> > >> Does someone know enough about this to figure out what to do? >> > > >> > > It seems to come from the call to __Pyx_GetBuffer, which is >> > > PyObject_GetBuffer in python 3. Maybe this is Python 3 not checking >> > > for an extent of 1, but instead only checking the stride (which is a >> > > multiple of the itemsize)? >> > >> > No, Py3 doesn't do any validation here, it does a straight call into the >> > object's slot function, i.e. into the NumPy array itself. >> > >> > So, the question is: who's wrong here? The test or NumPy? >> > >> > Hmm, or maybe just me. I didn't define the NPY_RELAXED_STRIDES_CHECKING >> > environment variable for the NumPy build. Let's try that first. >> > >> > http://docs.scipy.org/doc/numpy/release.html#npy-relaxed-strides-checking >> >> It's probably nicer though to write the Cython tests in such a way that they can pass against a default numpy installation? To avoid user confusion and all that. >> >> If those tests depend on numpy having relaxed strides enabled, then I'd suggest checking for this a test time and skipping the tests if not found. The relaxed strides docs give a recipe for determining how numpy was built. >> >> -n >> >> >> ___ >> cython-devel mailing list >> cython-devel@python.org >> https://mail.python.org/mailman/listinfo/cython-devel >> > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
Nathaniel Smith, 04.01.2014 18:36: > On 4 Jan 2014 17:07, "Robert Bradshaw" wrote: >> Yes, that'd be good. Do you know how? I won't have time to look at this > 'till next week. > > Something like > > if not np.ones((10, 1), order="C").flags.f_contiguous: > # numpy without relaxed stride support > raise SkipTest https://github.com/cython/cython/commit/e1982505564125714d2010391eecfb8de61626fa Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
Stefan Behnel, 04.01.2014 22:47: > Nathaniel Smith, 04.01.2014 18:36: >> On 4 Jan 2014 17:07, "Robert Bradshaw" wrote: >>> Yes, that'd be good. Do you know how? I won't have time to look at this >> 'till next week. >> >> Something like >> >> if not np.ones((10, 1), order="C").flags.f_contiguous: >> # numpy without relaxed stride support >> raise SkipTest > > https://github.com/cython/cython/commit/e1982505564125714d2010391eecfb8de61626fa Hmm, but this doesn't seem to work for me in older NumPy versions, although the original test used to work there. Should we explicitly test for NumPy 0.18+ as well? Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
Stefan Behnel, 04.01.2014 22:51: > Stefan Behnel, 04.01.2014 22:47: >> Nathaniel Smith, 04.01.2014 18:36: >>> On 4 Jan 2014 17:07, "Robert Bradshaw" wrote: Yes, that'd be good. Do you know how? I won't have time to look at this >>> 'till next week. >>> >>> Something like >>> >>> if not np.ones((10, 1), order="C").flags.f_contiguous: >>> # numpy without relaxed stride support >>> raise SkipTest >> >> https://github.com/cython/cython/commit/e1982505564125714d2010391eecfb8de61626fa > > Hmm, but this doesn't seem to work for me in older NumPy versions, although > the original test used to work there. Should we explicitly test for NumPy > 0.18+ as well? (Looks like the Cython versioning scheme starts messing up my sense for good versioning...) I meant NumPy 1.8, 'obviously', i.e. this: https://github.com/cython/cython/commit/a95d8f912c995300a13fc244ee71bc277668cb9a Stefan ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "relaxed_strides" test broken with NumPy 1.8
No, I'm missing something now; AFAIK there are only two numpy behaviors: with relaxed strides and without relaxed strides, and version number should be irrelevant beyond that. What's different between 1.8-without-relaxed-strides and 1.7 that makes the test break? On 4 Jan 2014 22:01, "Stefan Behnel" wrote: > Stefan Behnel, 04.01.2014 22:51: > > Stefan Behnel, 04.01.2014 22:47: > >> Nathaniel Smith, 04.01.2014 18:36: > >>> On 4 Jan 2014 17:07, "Robert Bradshaw" wrote: > Yes, that'd be good. Do you know how? I won't have time to look at > this > >>> 'till next week. > >>> > >>> Something like > >>> > >>> if not np.ones((10, 1), order="C").flags.f_contiguous: > >>> # numpy without relaxed stride support > >>> raise SkipTest > >> > >> > https://github.com/cython/cython/commit/e1982505564125714d2010391eecfb8de61626fa > > > > Hmm, but this doesn't seem to work for me in older NumPy versions, > although > > the original test used to work there. Should we explicitly test for NumPy > > 0.18+ as well? > > (Looks like the Cython versioning scheme starts messing up my sense for > good versioning...) > > I meant NumPy 1.8, 'obviously', i.e. this: > > > https://github.com/cython/cython/commit/a95d8f912c995300a13fc244ee71bc277668cb9a > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
