Re: [Rd] Another issue using multi-processing linear algebra libraries

2024-08-08 Thread Ivan Krylov via R-devel
В Wed, 7 Aug 2024 07:47:38 -0400 Dipterix Wang пишет: > I wonder if R initiates a system environment or options to instruct > the packages on the number of cores to use? A lot of thought and experience with various HPC systems went into availableCores(), a function from the zero-dependency 'para

[Rd] Another issue using multi-processing linear algebra libraries

2024-08-08 Thread Mervin Milton Fansler Iv
> "There’s a downside not mentioned in the manual that caught and baffled me > for a while. I was using all 64 cores of an AWS instance via > parallel::mclapply() and doing matrix multiplications in the parallelized > function. If the matrices were big enough the linked BLAS or LAPACK would try

Re: [Rd] Another issue using multi-processing linear algebra libraries

2024-08-07 Thread Dipterix Wang
I also have this questions for this. I wonder if R initiates a system environment or options to instruct the packages on the number of cores to use? It doesn't have to be mandatory for now, but at least package maintainers can have shared consensus and start to adopt this humble settings rather

Re: [Rd] Another issue using multi-processing linear algebra libraries

2024-08-06 Thread Ivan Krylov via R-devel
В Tue, 6 Aug 2024 10:19:25 -0400 Rob Steele via R-devel пишет: > Would it make sense to add a parameter somewhere, to mclapply(), say, > telling R to not use multiprocessing libraries? It would be great if we had a way to limit all kinds of multiprocessing (child processes, OpenMP threads, pthre

[Rd] Another issue using multi-processing linear algebra libraries

2024-08-06 Thread Rob Steele via R-devel
>From the R Installation and Admin manual: "There is a tendency for re-distributors of R to use ‘enhanced’ linear algebra libraries without explaining their downsides.” There’s a downside not mentioned in the manual that caught and baffled me for a while. I was using all 64 cores of an