Re: [Cython] Cython 0.16 and ndarray fields deprecation
On 29 February 2012 17:57, Dag Sverre Seljebotn wrote: > On 02/29/2012 09:42 AM, Stefan Behnel wrote: >> >> Dag Sverre Seljebotn, 29.02.2012 18:06: >>> >>> I'm wondering what the best course of action for deprecating the shape >>> field in numpy.pxd is. >>> >>> The thing is, currently "shape" really gets in the way. In most >>> situations >>> it is OK with slow access to shape through the Python layer, and >>> "arr.shape[0]" is often just fine, but currently one is in a situation >>> where one must either write "(arr).shape[0])" or >>> "np.PyArray_DIMS(arr)[0]", or be faced with code that isn't >>> forward-compatible with NumPy. >> >> >> Can Cython emulate this at the C layer? And even your work-around for the >> Python object access looks more like a Cython bug to me. I wouldn't know >> why that can't "just work". It usually works for other undeclared Python >> attributes of "anything", so it might just as well be made to work here. > > > Well, the problem is that shape is currently declared as a C field. It is > also available as a Python attribute. Usually the user doesn't care which > one is used, but the C field is declared for the few cases where access is > speed-critical. > > Though even with current NumPy, I find myself doing "print > (arr).shape" in order to get a tuple rather than a Py_ssize_t*... > > >>> It would really be good to do the transition as fast as possible, so that >>> all Cython code eventually becomes ready for upcoming NumPy releases. >> >> >> But it previously worked, right? It's just no longer supported in newer >> NumPy versions IIUC? If that's the case, deleting it would break otherwise >> working code. No-one forces you to switch to the latest NumPy version, >> after all, and certainly not right now. A warning is much better. > > > It previously worked, but it turns out that it was always frowned-upon. I > didn't know that when I added the fields, and it was a convenient way of > speeding things up... I would personally prefer either cdef nogil extension class properties (needs compiler support) or just special-casing in the compiler, which shouldn't be too hard I think. Warnings would be a first step, but the linkage of ndarray attributes to C attributes is really an implementation detail, so it's better to keep supporting the attributes correctly. > Dag > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16 and ndarray fields deprecation
On 03/01/2012 04:03 AM, mark florisson wrote: On 29 February 2012 17:57, Dag Sverre Seljebotn wrote: On 02/29/2012 09:42 AM, Stefan Behnel wrote: Dag Sverre Seljebotn, 29.02.2012 18:06: I'm wondering what the best course of action for deprecating the shape field in numpy.pxd is. The thing is, currently "shape" really gets in the way. In most situations it is OK with slow access to shape through the Python layer, and "arr.shape[0]" is often just fine, but currently one is in a situation where one must either write "(arr).shape[0])" or "np.PyArray_DIMS(arr)[0]", or be faced with code that isn't forward-compatible with NumPy. Can Cython emulate this at the C layer? And even your work-around for the Python object access looks more like a Cython bug to me. I wouldn't know why that can't "just work". It usually works for other undeclared Python attributes of "anything", so it might just as well be made to work here. Well, the problem is that shape is currently declared as a C field. It is also available as a Python attribute. Usually the user doesn't care which one is used, but the C field is declared for the few cases where access is speed-critical. Though even with current NumPy, I find myself doing "print (arr).shape" in order to get a tuple rather than a Py_ssize_t*... It would really be good to do the transition as fast as possible, so that all Cython code eventually becomes ready for upcoming NumPy releases. But it previously worked, right? It's just no longer supported in newer NumPy versions IIUC? If that's the case, deleting it would break otherwise working code. No-one forces you to switch to the latest NumPy version, after all, and certainly not right now. A warning is much better. It previously worked, but it turns out that it was always frowned-upon. I didn't know that when I added the fields, and it was a convenient way of speeding things up... I would personally prefer either cdef nogil extension class properties (needs compiler support) or just special-casing in the compiler, which shouldn't be too hard I think. Warnings would be a first step, but the linkage of ndarray attributes to C attributes is really an implementation detail, so it's better to keep supporting the attributes correctly. So you are saying we (somehow) stick with supporting "arr.shape[0]" in the future, and perhaps even support "print arr.shape"? (+ arr.dim, arr.strides). Exactly how we could figure out at PyCon. I'm anyway leaning towards deprecating arr.data, as it's too different from what the Python attribute does. Reason I'm asking is that I'm giving a talk on Saturday, and I don't want to teach people bad habits -- so we must figure out what the bad habits are :-) (I think this applies for the PyCon poster as well...) [1] PyData workshop at Google's offices in Mountain View; the event was open for all but now it is full with a long waiting list, which is why I didn't announce it. http://pydataworkshop.eventbrite.com/ Dag ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16 and ndarray fields deprecation
On 01.03.2012 17:18, Dag Sverre Seljebotn wrote: are saying we (somehow) stick with supporting "arr.shape[0]" in the future, and perhaps even support "print arr.shape"? (+ arr.dim, arr.strides). What if you just deprecate ndarray support completely, and just focus on memory views? Yes, you will break all Cython code in the world depending on ndarrays. But you will do that anyway by tempering with the interface. And as changes to the NumPy C API mandates a change to the interface, I see no reason to keep it. If you are going to break all code, then just do it completely. It is worse to stick to syntax bloat. There should not be multiple ways to do the same (like the zen of Python). Sturla ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16 and ndarray fields deprecation
On 01.03.2012 17:18, Dag Sverre Seljebotn wrote: I'm anyway leaning towards deprecating arr.data, as it's too different from what the Python attribute does. This should be preferred, I think &arr[0] or &arr[0] The latter is exacty what arr.data will currently do in Cython (but not in Python). But there is code in SciPy that depends on the arr.data attribute in Cython, such as cKDTree. Sturla ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16 and ndarray fields deprecation
On 03/01/2012 09:29 AM, Sturla Molden wrote: On 01.03.2012 17:18, Dag Sverre Seljebotn wrote: are saying we (somehow) stick with supporting "arr.shape[0]" in the future, and perhaps even support "print arr.shape"? (+ arr.dim, arr.strides). What if you just deprecate ndarray support completely, and just focus on memory views? Yes, you will break all Cython code in the world depending on ndarrays. But you will do that anyway by tempering with the interface. And as changes to the NumPy C API mandates a change to the interface, I see no reason to keep it. If you are going to break all code, then just do it completely. It is worse to stick to syntax bloat. There should not be multiple ways to do the same (like the zen of Python). Yeah, I proposed this on another thread as one of the options, but the support wasn't overwhelming at the time... About the scipy kcdTree issue, the SciPy process of generating Cython code manually when the code is written makes the problem slightly smaller... Dag ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16 and ndarray fields deprecation
On 01.03.2012 19:33, Dag Sverre Seljebotn wrote: Yeah, I proposed this on another thread as one of the options, but the support wasn't overwhelming at the time... I think it is worse to break parts of it, thus introducing bugs that might go silent for a long time. Rather deprecate the whole ndarray interface. Sturla ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] RuntimeWarning: numpy.dtype size changed when cimporting numpy
I have a largish cython module that outputs the following warnings
every time I import the compiled module.
__main__:1: RuntimeWarning: numpy.dtype size changed, may indicate
binary incompatibility
__main__:1: RuntimeWarning: numpy.flatiter size changed, may indicate
binary incompatibility
Is this something I should worry about? I've traced this warning gets
generated when a `cimport numpy as np` line is processed in my module.
When I comment out that line I don't get any warning. To recreate
this error I've created a cython with only that line as follows:
numpy_test.pyx:
cimport numpy as np
A barebones setup in `setup.py`:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(name='numpy_test',
version='tip',
cmdclass={'build_ext': build_ext},
ext_modules=[Extension(name="numpy_test", sources=['numpy_test.pyx'])]
)
Then I do a local build like this:
python setup.py build
cd into the build directory:
cd build/lib.linux-x86_64-2.6/
verify the .so is there:
$ ls
numpy_test.so
and import it and get the runtime warning:
$ python -c "import numpy_test"
-c:1: RuntimeWarning: numpy.dtype size changed, may indicate binary
incompatibility
-c:1: RuntimeWarning: numpy.flatiter size changed, may indicate binary
incompatibility
I'm running 64 bit Ubuntu 10.04, Python 2.6.6, and Cython 0.15.1.
Any help would be greatly appreciated!
Rich
--
Richard P. Sharp Jr.
Lead Software Developer
Natural Capital Project
Stanford University, U Minnesota, TNC, WWF
371 Serra Mall
Stanford, CA 94305
http://www.stanford.edu/~rpsharp/
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
