Re: [Rd] boneheaded BLAS questions
On 3/18/21 3:53 AM, Ben Bolker wrote: Thanks. I know it's supposed to Just Work (and I definitely appreciate all the work that's gone into making it Just Work 99% of the time!). I tried --with-lapack, no joy. Will try to decipher the rules file tomorrow ... This is documented in R Admin manual, section A.3, and there is also "configure --help". On my Ubuntu 20.04, using "--with-blas --with-lapack" when a BLAS/LAPACK implementation is installed via "apt" works for me: with libblas3, liblapack3 I get in R via sessionInfo() BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0 then I install libopenblas0 and get, after re-starting R (not rebuilding): BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 Best Tomas cheers Ben On 3/17/21 10:25 PM, Dirk Eddelbuettel wrote: Ben, This stuff has worked unchanged since the 1990s when we had a _really_ far sighted fellow in Debian come up with the 'switch the links' scheme which was (and is) subsequently deployed by many numerical applications within Debian, R and e.g. Octave included. And I used this ability to switch over a decade ago in a never-quite-finished paper which resulted in a package as well as a vignette as paper draft on CRAN: gcbd [1] It used the ability to switch between implementation to time and compare and benchmark the various BLAS and LAPACK libraries -- which was then motivated by a comparison with GPUs. (The actual code / package is stale-ish as some of the underlying packages have gone as eg the GPU one -- but the mechanics you are after still work the exact same way on Debian and derivarives including Ubuntu and PopOS.) (As a complete aside, the state of the art here is now one level up in libraries based on flame/blis (a riff on blas) which can do a similar logical switch _at runtime_ (rather than by flipping softlinks and restarting the app). Julia and some other languages uses that, I think Fedora may have it in its R build as well. Inaki may know more...) That said, from the top of my head, I think you error may just be with the second R compilation -- I always (i.e. for the Debian package) use both --with-blas --with-lapack and not just --with-blas. And I do there is public: if you know where to look you can see the exact invocation of the Debian build of the R package (which Ubuntu and Pop and ... then shadow) [2] Hth, Dirk [1] https://cran.r-project.org/package=gcbd [2] https://sources.debian.org/src/r-base/4.0.4-1/debian/rules/ (and I apologise for how messy this still is) __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] boneheaded BLAS questions
On 3/18/21 9:15 AM, Tomas Kalibera wrote: On 3/18/21 3:53 AM, Ben Bolker wrote: Thanks. I know it's supposed to Just Work (and I definitely appreciate all the work that's gone into making it Just Work 99% of the time!). I tried --with-lapack, no joy. Will try to decipher the rules file tomorrow ... This is documented in R Admin manual, section A.3, and there is also "configure --help". On my Ubuntu 20.04, using "--with-blas --with-lapack" when a BLAS/LAPACK implementation is installed via "apt" works for me: with libblas3, liblapack3 I get in R via sessionInfo() BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0 then I install libopenblas0 and get, after re-starting R (not rebuilding): BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 and to install say MKL, this works for me: apt-get install intel-mkl-full and then: env MKL_INTERFACE_LAYER=GNU,LP64 MKL_THREADING_LAYER=GNU R gives me: BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/libmkl_rt.so This is following documentation in A.3.1.3 of R Admin manual. Best Tomas Best Tomas cheers Ben On 3/17/21 10:25 PM, Dirk Eddelbuettel wrote: Ben, This stuff has worked unchanged since the 1990s when we had a _really_ far sighted fellow in Debian come up with the 'switch the links' scheme which was (and is) subsequently deployed by many numerical applications within Debian, R and e.g. Octave included. And I used this ability to switch over a decade ago in a never-quite-finished paper which resulted in a package as well as a vignette as paper draft on CRAN: gcbd [1] It used the ability to switch between implementation to time and compare and benchmark the various BLAS and LAPACK libraries -- which was then motivated by a comparison with GPUs. (The actual code / package is stale-ish as some of the underlying packages have gone as eg the GPU one -- but the mechanics you are after still work the exact same way on Debian and derivarives including Ubuntu and PopOS.) (As a complete aside, the state of the art here is now one level up in libraries based on flame/blis (a riff on blas) which can do a similar logical switch _at runtime_ (rather than by flipping softlinks and restarting the app). Julia and some other languages uses that, I think Fedora may have it in its R build as well. Inaki may know more...) That said, from the top of my head, I think you error may just be with the second R compilation -- I always (i.e. for the Debian package) use both --with-blas --with-lapack and not just --with-blas. And I do there is public: if you know where to look you can see the exact invocation of the Debian build of the R package (which Ubuntu and Pop and ... then shadow) [2] Hth, Dirk [1] https://cran.r-project.org/package=gcbd [2] https://sources.debian.org/src/r-base/4.0.4-1/debian/rules/ (and I apologise for how messy this still is) __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] boneheaded BLAS questions
On Thu, 18 Mar 2021 at 05:10, Dirk Eddelbuettel wrote: > > > On 17 March 2021 at 22:53, Ben Bolker wrote: > |Thanks. I know it's supposed to Just Work (and I definitely > | appreciate all the work that's gone into making it Just Work 99% of the > | time!). > > And for what it is worth, the aforementioned 'switching from within' solution > is using FlexiBLAS (not BLIS as I had said in the previous email), and was > described in an R application R here: > > > https://www.enchufa2.es/archives/switch-blas-lapack-without-leaving-your-r-session.html Thanks, Dirk. Yes, since Fedora 33 (current release), we leverage this excellent work by Martin Köhler et al. [1], so that every BLAS/LAPACK consumer in Fedora is linked against FlexiBLAS, which enables transparent live switching. And there are R and octave packages providing bindings, as shown in the post above. Julia is in fact the only component that is currently *not* using it due to the particularities of their BLAS/LAPACK stack management, but they are interested in FlexiBLAS too and some work is underway [2]. If you are interested in this, Ben, you could compile FlexiBLAS yourself, docs are very clear and it's pretty straightforward. And then you only need to tell R to link against libflexiblas. For that, as previously described, we use (see lines 691-693 in [3]): --with-lapack --with-blas=flexiblas A small tweak in the configure is required though (see line 679 in [3]; in fact, I should port a proper fix upstream, but I didn't find the time yet). And if you have any issue Martin or myself could help. [1] https://www.mpi-magdeburg.mpg.de/projects/flexiblas [2] https://github.com/mpimd-csc/flexiblas/issues/12 [3] https://src.fedoraproject.org/rpms/R/blob/rawhide/f/R.spec -- Iñaki Úcar __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] boneheaded BLAS questions
On 18 March 2021 at 09:15, Tomas Kalibera wrote: | This is documented in R Admin manual, section A.3, and there is also | "configure --help". | | On my Ubuntu 20.04, using "--with-blas --with-lapack" when a BLAS/LAPACK | implementation is installed via "apt" works for me: | | with libblas3, liblapack3 I get in R via sessionInfo() | | BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 | LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0 | | then I install libopenblas0 and get, after re-starting R (not rebuilding): | | BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 Yes, thank you, that is exactly what I use and recommend too. And ... On 18 March 2021 at 09:34, Tomas Kalibera wrote: | and to install say MKL, this works for me: | | apt-get install intel-mkl-full | | and then: | | env MKL_INTERFACE_LAYER=GNU,LP64 MKL_THREADING_LAYER=GNU R | | gives me: | | BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/libmkl_rt.so | | This is following documentation in A.3.1.3 of R Admin manual. ... which is what the (surprisingly popular, 139 stars) simple two-year old script at GitHub has been automated. https://github.com/eddelbuettel/mkl4deb I may need to update the recommendation for the two MKL_* variables. Dirk -- https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] boneheaded BLAS questions
For what it's worth I eventually got it to build in a hacky way (had to add -lopenblaslib manually). FWIW I *did* RTFM, several times, but for whatever reason the standard recipes are not working for me ... thanks! Ben Bolker On 3/18/21 7:52 AM, Dirk Eddelbuettel wrote: On 18 March 2021 at 09:15, Tomas Kalibera wrote: | This is documented in R Admin manual, section A.3, and there is also | "configure --help". | | On my Ubuntu 20.04, using "--with-blas --with-lapack" when a BLAS/LAPACK | implementation is installed via "apt" works for me: | | with libblas3, liblapack3 I get in R via sessionInfo() | | BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 | LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0 | | then I install libopenblas0 and get, after re-starting R (not rebuilding): | | BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 Yes, thank you, that is exactly what I use and recommend too. And ... On 18 March 2021 at 09:34, Tomas Kalibera wrote: | and to install say MKL, this works for me: | | apt-get install intel-mkl-full | | and then: | | env MKL_INTERFACE_LAYER=GNU,LP64 MKL_THREADING_LAYER=GNU R | | gives me: | | BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/libmkl_rt.so | | This is following documentation in A.3.1.3 of R Admin manual. ... which is what the (surprisingly popular, 139 stars) simple two-year old script at GitHub has been automated. https://github.com/eddelbuettel/mkl4deb I may need to update the recommendation for the two MKL_* variables. Dirk __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
