[Numpy-discussion] Re: deprecating numpy.distutils
On Sun, Jan 16, 2022 at 10:40 PM Sandro Tosi wrote:
> > 4. Write a docs page with migration info on how to deal with the
> deprecation, and update it before the 1.23.0 release.
>
> i think this will be extremely important. Just to give you the current
> POV of debian projects using numpy.distutils:
>
> -
> https://codesearch.debian.net/search?q=numpy.distutils+-package%3Anumpy&literal=0&perpkg=1
> (there are 74 of them)
> -
> https://codesearch.debian.net/search?q=from+numpy+import+.*distutils&literal=0
> (only one "from numpy import distutils")
>
> hope this could help you come up with a smooth deprecation process :)
>
Thanks Sandro, that is helpful. I had a look at all those 15 pages of
search results:
PyPy: just a few patches for numpy <1.11.2 which can be dropped
Silx: uses Configuration (i.e., nested setup.py's)
Electrum: just a few patches for numpy <1.11.2 which can be dropped
Setuptools: just a few patches for numpy <1.11.2 which can be dropped
python-hdf4: nothing special, can be switched to setuptools
pybigwig: uses `get_info('npymath')` to (optionally) build against numpy -
this may need to be added to setuptools
python-pot: uses private numpy.distutils compiler support; can be switched
to setuptools or vendored
petsc4py: small setup.py, but uses Fortran support and Configuration -
should probably switch build system
basemap: uses Extension, can just switch to setuptools
openexr: just queries numpy include dirs in an incorrect fashion, should
use `numpy.get_include`
rocketcea: nothing special, can just switch to setuptools
brian: no usage, just a string `numpy.distutils` in docs
libvigraimpex: just queries numpy include dirs in an incorrect fashion,
should use `numpy.get_include`
gpaw: some weird monkeypatching in shell scripts of numpy's BLAS/LAPACK
support, can probably just be dropped
pyferret: uses Extension, can just switch to setuptools
adolc: just queries numpy include dirs in an incorrect fashion, should use
`numpy.get_include`
theano: uses BLAS support, they should probably vendor `system_info.py` (or
switch build systems)
scikit-image: uses Configuration
scikit-ffm: uses Configuration, but only two setup.py files - easy to
consolidate in a single setup.py and switch to setuptools
pywavelets: no longer uses numpy.distutils
pybind11: just a code comment, no numpy.distutils usage
sfepy: uses Configuration
nibabel: only one usage in a test case, just drop it
versioneer: just a mention in the docs
cypari: just a patch for numpy < 1.11.2, can be dropped
blender: just a patch that can be dropped
scikit-learn: uses Configuration and build_ext modification for parallel
compilation
nipype: minor usage of Configuration, can be dropped
guiqwt: uses Extension but no longer has Fortran code, so can switch to
setuptools
The summary of this is: if Configuration is added to setuptools, 90% of
packages are fine switching to setuptools. Scikit-learn is planning to
follow SciPy and move to Meson. Theano is the only real consumer of
numpy.distutils BLAS/LAPACK support, which they should vendor if they don't
want to switch build systems. There are no other serious issues here.
I'm sure there are some more packages out there relying on Fortran support
in particular, they are best off moving to Meson or CMake.
Cheers,
Ralf
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com
[Numpy-discussion] Re: deprecating numpy.distutils
On Mon, 17 Jan 2022 at 10:01, Ralf Gommers wrote: > > > On Sun, Jan 16, 2022 at 10:40 PM Sandro Tosi > wrote: > >> > 4. Write a docs page with migration info on how to deal with the >> deprecation, and update it before the 1.23.0 release. >> >> i think this will be extremely important. Just to give you the current >> POV of debian projects using numpy.distutils: >> >> - >> https://codesearch.debian.net/search?q=numpy.distutils+-package%3Anumpy&literal=0&perpkg=1 >> (there are 74 of them) >> - >> https://codesearch.debian.net/search?q=from+numpy+import+.*distutils&literal=0 >> (only one "from numpy import distutils") >> >> hope this could help you come up with a smooth deprecation process :) >> > > Thanks Sandro, that is helpful. I had a look at all those 15 pages of > search results: > ... > Theano is the only real consumer of numpy.distutils BLAS/LAPACK support, > which they should vendor if they don't want to switch build systems. > Note that Theano is no longer maintained as seen at the top of the README: https://github.com/Theano/Theano Forked development continues under the name Aesara: https://github.com/aesara-devs/aesara Not sure if Aesara differs in respect of using numpy.distutils but Theano is 2 years out of date in comparison. -- Oscar ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: deprecating numpy.distutils
On Mon, Jan 17, 2022 at 11:27 AM Oscar Benjamin wrote: > On Mon, 17 Jan 2022 at 10:01, Ralf Gommers wrote: > >> >> >> On Sun, Jan 16, 2022 at 10:40 PM Sandro Tosi >> wrote: >> >>> > 4. Write a docs page with migration info on how to deal with the >>> deprecation, and update it before the 1.23.0 release. >>> >>> i think this will be extremely important. Just to give you the current >>> POV of debian projects using numpy.distutils: >>> >>> - >>> https://codesearch.debian.net/search?q=numpy.distutils+-package%3Anumpy&literal=0&perpkg=1 >>> (there are 74 of them) >>> - >>> https://codesearch.debian.net/search?q=from+numpy+import+.*distutils&literal=0 >>> (only one "from numpy import distutils") >>> >>> hope this could help you come up with a smooth deprecation process :) >>> >> >> Thanks Sandro, that is helpful. I had a look at all those 15 pages of >> search results: >> > ... > >> Theano is the only real consumer of numpy.distutils BLAS/LAPACK support, >> which they should vendor if they don't want to switch build systems. >> > > Note that Theano is no longer maintained as seen at the top of the README: > https://github.com/Theano/Theano > > Forked development continues under the name Aesara: > https://github.com/aesara-devs/aesara > > Not sure if Aesara differs in respect of using numpy.distutils but Theano > is 2 years out of date in comparison. > Thanks for pointing that out Oscar. Still the same situation, one nontrivial use to query for BLAS config: https://github.com/aesara-devs/aesara/blob/4468ba601f96d82a387ab0d8ab09bdedbf9bf49a/aesara/link/c/cmodule.py#L2634 Cheers, Ralf ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: deprecating numpy.distutils
Hi,
I should mention many of our software projects (fabio, pyFAI, ...) rely
on silx for the nested setup.py part, and not all are mentionned in this list.
In the absence of alternative solution, we limit the version of
setup.py for now but we are willing to help in finding a solution (our
complete build stack would collapses).
Well the parallel build feature is a very much appreciated because the
serial compilation of our project take dozens of minutes and our
computers are HPC nodes with plenty of cores...
I find funny the way of first deprecate ... then look if any
alternative solution exists.
If I understand properly, your idea is to port the nested setup.py part
from numpy.distutils to setuptools ? Are people of setuptools already in the
loop ?
Cheers,
Jerome
On Mon, 17 Jan 2022 10:57:40 +0100
Ralf Gommers wrote:
> On Sun, Jan 16, 2022 at 10:40 PM Sandro Tosi wrote:
>
> > > 4. Write a docs page with migration info on how to deal with the
> > deprecation, and update it before the 1.23.0 release.
> >
> > i think this will be extremely important. Just to give you the current
> > POV of debian projects using numpy.distutils:
> >
> > -
> > https://codesearch.debian.net/search?q=numpy.distutils+-package%3Anumpy&literal=0&perpkg=1
> > (there are 74 of them)
> > -
> > https://codesearch.debian.net/search?q=from+numpy+import+.*distutils&literal=0
> > (only one "from numpy import distutils")
> >
> > hope this could help you come up with a smooth deprecation process :)
> >
>
> Thanks Sandro, that is helpful. I had a look at all those 15 pages of
> search results:
>
> PyPy: just a few patches for numpy <1.11.2 which can be dropped
> Silx: uses Configuration (i.e., nested setup.py's)
> Electrum: just a few patches for numpy <1.11.2 which can be dropped
> Setuptools: just a few patches for numpy <1.11.2 which can be dropped
> python-hdf4: nothing special, can be switched to setuptools
> pybigwig: uses `get_info('npymath')` to (optionally) build against numpy -
> this may need to be added to setuptools
> python-pot: uses private numpy.distutils compiler support; can be switched
> to setuptools or vendored
> petsc4py: small setup.py, but uses Fortran support and Configuration -
> should probably switch build system
> basemap: uses Extension, can just switch to setuptools
> openexr: just queries numpy include dirs in an incorrect fashion, should
> use `numpy.get_include`
> rocketcea: nothing special, can just switch to setuptools
> brian: no usage, just a string `numpy.distutils` in docs
> libvigraimpex: just queries numpy include dirs in an incorrect fashion,
> should use `numpy.get_include`
> gpaw: some weird monkeypatching in shell scripts of numpy's BLAS/LAPACK
> support, can probably just be dropped
> pyferret: uses Extension, can just switch to setuptools
> adolc: just queries numpy include dirs in an incorrect fashion, should use
> `numpy.get_include`
> theano: uses BLAS support, they should probably vendor `system_info.py` (or
> switch build systems)
> scikit-image: uses Configuration
> scikit-ffm: uses Configuration, but only two setup.py files - easy to
> consolidate in a single setup.py and switch to setuptools
> pywavelets: no longer uses numpy.distutils
> pybind11: just a code comment, no numpy.distutils usage
> sfepy: uses Configuration
> nibabel: only one usage in a test case, just drop it
> versioneer: just a mention in the docs
> cypari: just a patch for numpy < 1.11.2, can be dropped
> blender: just a patch that can be dropped
> scikit-learn: uses Configuration and build_ext modification for parallel
> compilation
> nipype: minor usage of Configuration, can be dropped
> guiqwt: uses Extension but no longer has Fortran code, so can switch to
> setuptools
>
>
> The summary of this is: if Configuration is added to setuptools, 90% of
> packages are fine switching to setuptools. Scikit-learn is planning to
> follow SciPy and move to Meson. Theano is the only real consumer of
> numpy.distutils BLAS/LAPACK support, which they should vendor if they don't
> want to switch build systems. There are no other serious issues here.
>
> I'm sure there are some more packages out there relying on Fortran support
> in particular, they are best off moving to Meson or CMake.
>
> Cheers,
> Ralf
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com
[Numpy-discussion] Re: deprecating numpy.distutils
On Mon, Jan 17, 2022 at 3:17 PM Jerome Kieffer
wrote:
> Hi,
>
> I should mention many of our software projects (fabio, pyFAI, ...) rely
> on silx for the nested setup.py part, and not all are mentionned in this
> list.
> In the absence of alternative solution, we limit the version of
> setup.py for now but we are willing to help in finding a solution (our
> complete build stack would collapses).
>
> Well the parallel build feature is a very much appreciated because the
> serial compilation of our project take dozens of minutes and our
> computers are HPC nodes with plenty of cores...
>
Thanks for pointing those projects out Jerome. It looks like you'll be fine
moving to setuptools if Configuration and parallel compilation get added
there. Although with those kinds of build times you may at some point want
to move - Meson is significantly faster.
> I find funny the way of first deprecate ... then look if any
> alternative solution exists.
>
Well, it's morer less just formalizing the status quo. `distutils` is going
away, so if we do nothing then `numpy.distutils` simply also goes away.
This is basically just (a) a heads up to users what is coming, and (b)
giving people who need features and don't want to move build systems the
time to consider pitching in with moving the things they need into
setuptools.
> If I understand properly, your idea is to port the nested setup.py part
> from numpy.distutils to setuptools ? Are people of setuptools already in
> the loop ?
>
Yes, the lead setuptools maintainer has stated that they're happy to accept
any new features that make sense.
Cheers,
Ralf
> Cheers,
>
> Jerome
>
> On Mon, 17 Jan 2022 10:57:40 +0100
> Ralf Gommers wrote:
>
> > On Sun, Jan 16, 2022 at 10:40 PM Sandro Tosi
> wrote:
> >
> > > > 4. Write a docs page with migration info on how to deal with the
> > > deprecation, and update it before the 1.23.0 release.
> > >
> > > i think this will be extremely important. Just to give you the current
> > > POV of debian projects using numpy.distutils:
> > >
> > > -
> > >
> https://codesearch.debian.net/search?q=numpy.distutils+-package%3Anumpy&literal=0&perpkg=1
> > > (there are 74 of them)
> > > -
> > >
> https://codesearch.debian.net/search?q=from+numpy+import+.*distutils&literal=0
> > > (only one "from numpy import distutils")
> > >
> > > hope this could help you come up with a smooth deprecation process :)
> > >
> >
> > Thanks Sandro, that is helpful. I had a look at all those 15 pages of
> > search results:
> >
> > PyPy: just a few patches for numpy <1.11.2 which can be dropped
> > Silx: uses Configuration (i.e., nested setup.py's)
> > Electrum: just a few patches for numpy <1.11.2 which can be dropped
> > Setuptools: just a few patches for numpy <1.11.2 which can be dropped
> > python-hdf4: nothing special, can be switched to setuptools
> > pybigwig: uses `get_info('npymath')` to (optionally) build against numpy
> -
> > this may need to be added to setuptools
> > python-pot: uses private numpy.distutils compiler support; can be
> switched
> > to setuptools or vendored
> > petsc4py: small setup.py, but uses Fortran support and Configuration -
> > should probably switch build system
> > basemap: uses Extension, can just switch to setuptools
> > openexr: just queries numpy include dirs in an incorrect fashion, should
> > use `numpy.get_include`
> > rocketcea: nothing special, can just switch to setuptools
> > brian: no usage, just a string `numpy.distutils` in docs
> > libvigraimpex: just queries numpy include dirs in an incorrect fashion,
> > should use `numpy.get_include`
> > gpaw: some weird monkeypatching in shell scripts of numpy's BLAS/LAPACK
> > support, can probably just be dropped
> > pyferret: uses Extension, can just switch to setuptools
> > adolc: just queries numpy include dirs in an incorrect fashion, should
> use
> > `numpy.get_include`
> > theano: uses BLAS support, they should probably vendor `system_info.py`
> (or
> > switch build systems)
> > scikit-image: uses Configuration
> > scikit-ffm: uses Configuration, but only two setup.py files - easy to
> > consolidate in a single setup.py and switch to setuptools
> > pywavelets: no longer uses numpy.distutils
> > pybind11: just a code comment, no numpy.distutils usage
> > sfepy: uses Configuration
> > nibabel: only one usage in a test case, just drop it
> > versioneer: just a mention in the docs
> > cypari: just a patch for numpy < 1.11.2, can be dropped
> > blender: just a patch that can be dropped
> > scikit-learn: uses Configuration and build_ext modification for parallel
> > compilation
> > nipype: minor usage of Configuration, can be dropped
> > guiqwt: uses Extension but no longer has Fortran code, so can switch to
> > setuptools
> >
> >
> > The summary of this is: if Configuration is added to setuptools, 90% of
> > packages are fine switching to setuptools. Scikit-learn is planning to
> > follow SciPy and move to Meson. Theano is the only real consumer of
> > numpy.distutils BLA
