Hi, everyone. TL;DR: I'm proposing removing eselect-blas/lapack and the virtuals, and using sci-libs/flexiblas instead.
1. The problem ============== The current Gentoo infrastructure for supporting multiple BLAS/LAPACK variants doesn't really work. The primary problem is that while the interfaces are standardized, they are also versioned and different providers (and their different versions) tend to diverge over what's implemented. On top of that, packages generally don't stop at a kind of "common subset" (whatever that would be), but tend to detect which routines are implemented by the BLAS/LAPACK libraries actually used. The end result is that if you build a package against library A, then it may or may not work with library B after switching. While I agree in principle with eselect-ldso being superior to regular alternatives, it suffers from this problem even more. Since we're building packages against sci-libs/lapack, i.e. the reference implementation, we're effectively building against the most complete implementation. Given the ABI problem, no solution based on alternatives or virtuals can really work. 2. Proposed solution ==================== I'd like to propose a two-tier replacement to the current system: 1. Base BLAS/LAPACK support via sci-libs/flexiblas. 2. USE flags for specific implementations for packages that use additional functions beyond standard BLAS/LAPACK interfaces. 2.1. sci-libs/flexiblas ----------------------- FlexiBLAS is a thin wrapper over BLAS/LAPACK that provides runtime- switchable backends. It's similar in principle to media-libs/libglvnd. It provides a modern complete BLAS/LAPACK interface that other packages can build on. At runtime, it dispatches calls to the selected provider, or if it doesn't implement the call in question, to sci-libs/lapack, or a built-in fallback. We get the best of both worlds: packages can use the complete API, we use the preferred provider when we can, and fall back when we can't. On top of that, unlike with eselect, the preferred provider can be overriden via user configuration and environment variables too. On the minus side, FlexiBLAS currently doesn't support the mixed LP64/ILP64 interface provided by the latest sci-libs/lapack (upstream is planning to add it). So we're stuck with LP64, unless we add USE=64bit- index to sci-libs/lapack, for the more traditional ILP64 interface. Another potential concern is that FlexiBLAS seems to be largely university-developed software, with a single maintainer, no public bug tracker and a semi-public code repository (it seems that only releases are pushed there rather than all changes). The author seems quite friendly, and addressed issues I've reported so far, but the risk is there. However, we're not alone here: Fedora has switched to FlexiBLAS already. 2.2. USE flags -------------- I also propose adding USE flags to packages whenever this makes sense. In particular, this could be useful for packages that e.g. use the additional functions MKL provides. I'm opposed to adding USE flags for different BLAS/LAPACK providers all over the place, since most of the time these flags would make no real difference and only harm the ability to dynamically dispatch. Of course, I do realize that some people will inevitably add unnecessary flags. They probably already did. 3. The transition ================= A major problem with this is transitioning from the current model to FlexiBLAS. I can think of two different approaches to that: 1. The hard approach -- we migrate packages one-by-one to use FlexiBLAS in place of the virtuals. 2. The lazy approach (untested) -- we replace BLAS/LAPACK libraries with symlinks to FlexiBLAS. 3.1. The hard approach ---------------------- The advantage of the hard approach is that it's clean and explicit. We update packages to link to libflexiblas explicitly. Other BLAS/LAPACK providers, including sci-libs/lapack, are left as-is, i.e. following upstream install. While transitioning packages, we test them, and eventually we remove the virtuals and eselect modules. Bad news is that: - it's a lot of work - a lot of packages will require patching to use flexiblas - people will keep mistakenly linking to sci-libs/lapack directly This is the approach Fedora took. 3.2. The lazy approach ---------------------- The rough idea is that we move sci-libs/lapack away, via renaming the libraries or moving them into a subdirectory, and put symlinks to flexiblas in their place. Packages can continue searching for libblas, liblapack, etc. as usual, and they will end up being linked to flexiblas instead. Note that I haven't found time to test this yet, so I may be missing some problem with it, but I can't think of one right now. What's unclear to me is how we handle pkg-config files and CMake: if we also replace them with symlinks to flexiblas ones, or leave as-is but with the original paths that are now replaced by symlinks. I also need to check if the CMake files installed by sci-libs/lapack are actually used at all. Bad news is that it's less clean, and leaves us forever diverging in how sci-libs/lapack is installed, and apps linking to '-lblas -lcblas -llapack -llapacke' instead of '-lflexiblas'. -- Best regards, Michał Górny
signature.asc
Description: This is a digitally signed message part
