Dear Henrik,
Thank you! This is quite helpful, especially your longer blog post.
Best regards,
Ravi
________________________________
From: Henrik Bengtsson <henrik.bengts...@gmail.com>
Sent: Sunday, July 2, 2023 4:33 AM
To: Ravi Varadhan <ravi.varad...@jhu.edu>
Cc: R-Help <r-help@r-project.org>
Subject: Re: [R] Number of Cores limited to two in CRAN
External Email - Use Caution
Short answer: You don't want to override that limit in your R package.
Don't do it.
Long answer: You'll find the reason for this in the 'CRAN Repository
Policy'
(https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fweb%2Fpackages%2Fpolicies.html&data=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311718317%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KDB6%2BqMgn4KrKtVEgWezfRY9isT926wLwZGg5yuDKI4%3D&reserved=0<https://cran.r-project.org/web/packages/policies.html>).
Specifically, the following passage:
"Checking the package should take as little CPU time as possible, as
the CRAN check farm is a very limited resource and there are thousands
of packages. Long-running tests and vignette code can be made optional
for checking, but do ensure that the checks that are left do exercise
all the features of the package.
**If running a package uses multiple threads/cores it must never use
more than two simultaneously: the check farm is a shared resource and
will typically be running many checks simultaneously.**
Examples should run for no more than a few seconds each: they are
intended to exemplify to the would-be user how to use the functions in
the package."
Basically, you can use two cores to demonstrate or validate (e.g. in
package tests) that your code *can* run in parallel, but you must not
use more than that to demonstrate that your code can "run super fast".
Even-longer answer: See my blog post 'Please Avoid detectCores() in
your R Packages'
(https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jottr.org%2F2022%2F12%2F05%2Favoid-detectcores%2F&data=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311718317%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xPkA6JCccHZcST76lWa6xFYWWKBOZYQCOezF5t4YVPM%3D&reserved=0<https://www.jottr.org/2022/12/05/avoid-detectcores/>)
from 2022-12-05 for even more reasons.
/Henrik
On Sun, Jul 2, 2023 at 9:55 AM Ravi Varadhan via R-help
<r-help@r-project.org> wrote:
>
> This is the specific error messsage from R CMD check --as-cran
>
>
> Error in .check_ncores(length(names)) : 16 simultaneous processes spawned
> Calls: prepost -> makeCluster -> makePSOCKcluster -> .check_ncores
> Execution halted
>
>
> Thanks,
> Ravi
>
> ________________________________
> From: Ravi Varadhan
> Sent: Saturday, July 1, 2023 1:15 PM
> To: R-Help <r-help@r-project.org>
> Subject: Number of Cores limited to two in CRAN
>
> Hi,
> I am developing a package where I would like to utilize multiple cores for
> parallel computing. However, I get an error message when I run R CMD check
> --as-cran.
>
> I read that CRAN limits the number of cores to 2. Is this correct? Is there
> any way to overcome this limitation?
>
> Thank you,
> Ravi
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-help&data=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311873942%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KTNwXurNvbEhviVf15RF0la%2FvUVGBW60yuyVVy9Umxc%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-help>
> PLEASE do read the posting guide
> https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.r-project.org%2Fposting-guide.html&data=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311873942%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HvGb20UiUey9O3plynLiRmHQtqpjfrndk2iPfvleT3Q%3D&reserved=0<http://www.r-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
