This may be more of a question for r-package-devel than for r-devel.
On 23 November 2021 at 14:11, Dipterix Wang wrote:
| I recently received an email from Prof. Ripley. He pointed out that my
package seriously violates the CRAN policy: "using 8 threads is a serious
violation of the CRAN policy”. By default the number of cores my package uses
is determined from system CPU cores. After carefully reading all the CRAN
policies, now I understand that CRAN does not allow a package to use more than
2 CPU cores when checking a package. I can easily change my code to let my
tests comply to that constraint.
|
| However, this warning worries me because my package uses OpenMP. I got
“caught" partially because I printed the number of cores used in the package
startup message, and one of my test exceeded the time limit (which leads to
manual inspection). However, what if I develop a package that imports on those
openmp-dependent packages? (For example, data.table, fst…) These packages use
more than 2 cores by default. If not carefully treated, it’ll be very easy to
exceed that limit, and it’s very hard for CRAN to detect it.
|
| Is there any reliable way to let packages to know if they are on CRAN, so
they can set omp cores to 2 by default?
The question is valid (CRAN runs tests on many packages at once, we cannot
hog machines from a single package) and interesting: how can we give our
users all the horsepower they want and expect?
The available documentation (WRE, CRAN Policy, R source code, ...) offers
some hints so you could use n_threads = getOption("Ncpus", 1L) which means
that those who set this variable will get it.
For the tiledb package, we did something else and let the package do what the
underlying library does ("all cores", users expect perfomance), but also
added a function to throttle to what CRAN allows --- and then use that
function in all the examples, and the tests. There are a number of ways in
which the actual throttling is affected as it look at Ncpus (mentioned above)
as well as at OMP_THREAD_LIMIT which can be used to control OpenMP.
All that means that in the code CRAN runs -- via entry points from examples
and tests -- usage is throttled as CRAN desires. But all other ("normal") use
is "full throttle" and unlimited as users expect.
Hope this helps, Dirk
--
https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
