Hi,
Are blas providers really binary compatible? (ie exactly the same ABI?)
Nonetheless, you can be interested into looking at the mechanisms
used by OpenMP to load various OpenMP implementation. Their solution
uses a wrapper (the real (ie ELF library) libopencl) that is only a
ICD loader and various ICD (real implementations that are loaded
(and choosen) at runtime.
The application can explicitly choose the implementation, or a default
one can be used, or the user (though envvar) can choose/force an
implementation.
This path will need more works for BLAS/LAPACK maintainers. But, in
the long term, it would be probably better. If you are interested by
this, I would be pleased to discuss the design with you.
Regards,
Vincent
Le 20/05/2020 à 16:58, Mo Zhou a écrit :
> Hi fellow devs,
>
> I've suddenly got some inspiration on this problem, which resulted in a
> much better solution for the problem the original proposal confronts.
>
> I like this overhauled solution.
>
> No extra shared libs, no extra SONAMEs. No extra burden for the
> BLAS/LAPACK maintainers. Minor patching work for maintainers with
> special demands.
>
> --- New solution ---
>
> 1. BLAS providers create new directory under
> /usr/lib/<triplet>/libblas-<threading>/
> and put another copy of their alternative symlinks into the
> directories.
>
> e.g. libopenblas-pthread provides the extra symlink:
> /usr/lib/<triplet>/libblas-pthread/libblas.so{,.3}
>
> 2. Maintainers of reverse dependencies, when they have specific
> requirement on the threading implemetation, add the
> /usr/lib/<triplet>/libblas-<threading>/
> directory to the RPATH property of the resulting ELF binaries.
> In that way the programs can use the BLAS implementation without
> worring about the uncertainty of the alternatives configuration.
>
> Other packages insensitive to the threading implementations will
> use the libblas.so.3 in the global scope:
> /usr/lib/<triplet>/libblas.so.3 (a symlink)
>
> --- end solution ---
>
> That's it, much simpler and much more efficient than my first version.
>
> Comments please?
>
> On Wed, May 13, 2020 at 10:54:00PM +0200, Gard Spreemann wrote:
>> Mo Zhou <lu...@debian.org> writes:
>>
>>> Please comment:
>>> 1. Do we have a better solution where we can retain high performance and
>>> avoid threading trouble at the same time?
>>> 2. If we don't have a better solution, is my proposal acceptable?
>>> 3. In which way can my proposal be improved?
>>
>> Hi Mo,
>>
>> Your proposal seems sane to be, and a real service to the users!
>
> :-)
>
>> However, I am a little bit worried about the maintenance burden you are
>> setting up for the future; may it be promising too much to the users to
>> provide every single combination? Since this is BLAS, one could imagine
>> adding yet another couple of versions for each of the options, targeting
>> ever fancier vector instructions, etc. – at what point are special needs
>> best left to the user to cover?
>
> With the second version of solution, we can elegantly solve all the
> problem, and prevent the threading trouble from propagating to our end
> users!
>
>> I wish I had more constructive criticism. Thanks for the work!
>
> Let me do the critism: the first version of my solution is rubbish ...
>
>> An aside: do autopkgtest or other CIs currently test packages with
>> varied combinations of BLAS/LAPACK providers?
>
> No. But that's easy to implement.
>
> For example, we can add the following test cases in the autopkgtest
> control file of openblas:
>
> Test-Command: run-numpy-unit-test.sh
> Depends: libopenblas-pthread-dev
>
> Test-Command: run-numpy-unit-test.sh
> Depends: libopenblas-openmp-dev
>
> Test-Command: run-numpy-unit-test.sh
> Depends: libopenblas-serial-dev
>
> In this way we BLAS/LAPACK maintainers can immediately spot regressions
> in important packages. (adding these tests seems like my work)
>
>> Best,
>> Gard
>
--
Vincent Danjean GPG key ID 0xD17897FA vdanj...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo: deb http://people.debian.org/~vdanjean/debian unstable main
