[Numpy-discussion] Question regarding transitioning native extension to NumPy 2.0
I am working to transition mkl_fft and mkl_random to NumPy 2.0. Both of these projects contain native extensions. I have distilled unexpected behavior behind observed test failures in minimal C extension: https://github.com/oleksandr-pavlyk/reproducer-for-question-about-numpy2 The extension defines a single Python function which does expects numpy.ndarray, and queries its itemsize in two ways 1. By calling C function declared in "_aux.h" defined in "_aux.c" to call PyArray_ITEMSIZE and return the result 2. By calling PyArray_ITEMSIZE directly https://github.com/oleksandr-pavlyk/reproducer-for-question-about-numpy2/blob/main/ext.c#L19-L22 The result obtained by calling C function is always 0, while direct call gives the correct result. I am hoping for advice about what is wrong and how to fix it. Thank you, Sasha ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] updating the NumPy roadmap
Hi all, Now that the dust has settled on what we're including in the NumPy 2.0 release, it felt like a good time to update the project roadmap. After a few discussions with other maintainers I opened https://github.com/numpy/numpy/pull/26505. Part of it is a regular maintenance update: remove what we've implemented or decided not to do, and rewrite existing roadmap entries to reflect their current state. The other part is adding some new items. In particular: 1. Under the documentation section, add that we plan to make all example code interactive via jupyterlite-sphinx 2. Under the "platform support" section, add that we aim to start supporting free-threaded CPython, and plan to better define platform support tiers 3. A new section " binary size reduction" 4. A new section "NumPy 2.0 stabilization & downstream usage" 5. A new section "Security" (focused on supply chain security) If you are interested in what will end up on the roadmap, please do review that PR. For major topics, this thread can be used. New ideas are of course welcome, in particular from NumPy team members and from contributors who plan to work on something large enough that it should be represented on the roadmap. As a reminder: the roadmap has no dates, and not everything on it is guaranteed to materialize. The first sentences of the roadmap explain the purposes: "This is a live snapshot of tasks and features we will be investing resources in. It may be used to encourage and inspire developers and to search for funding." We plan to keep the PR open for at least 10 days from now, or until discussion settles. Cheers, Ralf ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Question regarding transitioning native extension to NumPy 2.0
Hi, The issue is caused by improperly importing the numpy C API. If you apply this diff, it will work: diff --git a/_aux.c b/_aux.c index e3f8f32..435b612 100644 --- a/_aux.c +++ b/_aux.c @@ -1,4 +1,6 @@ #include "Python.h" +#define NO_IMPORT_ARRAY +#define PY_ARRAY_UNIQUE_SYMBOL ExtModule #include "numpy/arrayobject.h" #include "_aux.h" diff --git a/ext.c b/ext.c index 65ad2c2..0e8eb3e 100644 --- a/ext.c +++ b/ext.c @@ -1,5 +1,6 @@ #define PY_SSIZE_T_CLEAN #include "Python.h" +#define PY_ARRAY_UNIQUE_SYMBOL ExtModule #include "numpy/arrayobject.h" #include "_aux.h" See also this new docs page, which hopefully clarifies this sort of arcane point: https://numpy.org/devdocs/reference/c-api/array.html#including-and-importing-the-c-api We were a bit loose in what we allowed before, effectively leaking details of the numpy C API. We cleaned that up, but that means C extensions now need to do this import dance correctly. Hope that helps, Nathan On Fri, May 24, 2024 at 12:52 PM Pavlyk, Oleksandr < oleksandr.pav...@intel.com> wrote: > I am working to transition mkl_fft and mkl_random to NumPy 2.0. > > Both of these projects contain native extensions. > > > > I have distilled unexpected behavior behind observed test failures in > minimal C extension: > > > > https://github.com/oleksandr-pavlyk/reproducer-for-question-about-numpy2 > > > > The extension defines a single Python function which does expects > numpy.ndarray, and queries its itemsize in two ways > > > >1. By calling C function declared in “_aux.h” defined in “_aux.c” to >call PyArray_ITEMSIZE and return the result >2. By calling PyArray_ITEMSIZE directly > > > > https://github.com/oleksandr-pavlyk/reproducer-for-question-about-numpy2/blob/main/ext.c#L19-L22 > > > > The result obtained by calling C function is always 0, while direct call > gives the correct result. > > > > I am hoping for advice about what is wrong and how to fix it. > > > > Thank you, > Sasha > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: nathan12...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
