Hi, I'm cc'ing debian-cross for the Multi-Arch related parts below.
* Dima Kogan <dko...@debian.org> [2024-09-07 12:59]:
I see the python3-numpy-dev package separates ALL the headers into /usr/lib/ARCH/. 99% of them are identical across arches though, so that's overkill, but it just costs our users a bit of extra disk space, so it's probably ok.Unfortunately, upstream designed its build API, in particular numpy.get_include(), for a *single* include directory. Unless we want to make NumPy headers globally visible in /usr/include and /usr/include/ARCH (which would cause problems with virtualenv isolation), we need to keep all headers in the same location.
The f2py source files ended up in -dev because a lot of other packages (including SciPy and even parts of NumPy itself) use numpy.get_include() + "/../../f2py/src" to find them.
(Note: I'm using the Debian terminology for host und build architecture)What's your thought about using :allowed/:any the way you noted above? That will allow a native python3-numpy and a foreign python3-numpy-dev to be installed, which most of the time isn't useful. I definitely may be missing some nuances here.Package: python3-numpy Architecture: any Multi-Arch: allowed Depends: python3-numpy-dev and Package: python3-numpy-dev Architecture: any Multi-Arch: same
With "M-A: allowed", the ":any" qualifier behaves pretty much like ":native" in Build-Depends. This is exactly what we want: we can have an executable NumPy on the build architecture with ":any" or ":native" and get the regular host architecture version otherwise.
It is not strictly necessary to have "M-A: allowed" for the B-D use-case, but it signals to package users that NumPy supports cross-building. Also, you cannot use ":native" for binary package dependencies, though that is probably not very relevant here, and there is prior art for "M-A: allowed" in the Python ecosystem, such as python3-dev and the Python interpreter itself.
We ship .a files in python3-numpy-dev. I don't know what those do. Any idea? It'd be good to test those.Those are support libraries to deal with older compilers and non-POSIX operationg systems. I know that SciPy links against npymath and the static libraries total less than 200K in size, so I will just keep them installed for the time being and totally ignore their existence otherwise.
The cross build support relies on pkg-config to select the correct host architecture via "/usr/lib/ARCH/pkgconfig/numpy.pc", so I patched all relevant API entrypoints (numpy-config, numpy.get_include()) to query $PKG_CONFIG or pkgconf. As long as cross-builders stick to the documented API to query the correct NumPy include path, everything will work out-of-the-box.python3-numpy ships arch-less symlinks: /usr/lib/python3/dist-packages/numpy/_core/include -> ../../../../x86_64-linux-gnu/python3-numpy/numpy/_core/include /usr/lib/python3/dist-packages/numpy/_core/lib -> ../../../../x86_64-linux-gnu/python3-numpy/numpy/_core/lib /usr/lib/python3/dist-packages/numpy/f2py/src -> ../../../../x86_64-linux-gnu/python3-numpy/numpy/f2py/src /usr/lib/python3/dist-packages/numpy/random/lib -> ../../../../x86_64-linux-gnu/python3-numpy/numpy/random/libI guess this preserves backwards compatibility, but could it cause confusion for cross-builders? Do we know what will break if we get rid of the symlinks and only make the new paths available?
However, I don't want python3-numpy to have a hard dependency on pkg-config, so the original code (which needs the original paths to exist) still runs as fallback. Besides, I'd rather be conservative here; there is plenty of NumPy code out there, and scientists can be quite... creative when it comes to API (ab-)use.
Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭────────────────────────────────────────────────────╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄⠀⠀⠀⠀ ╰────────────────────────────────────────────────────╯
signature.asc
Description: PGP signature
