Re: [R-pkg-devel] Vignette timing out on test server
On Wed, 17 Apr 2019 at 02:41, Roy Mendelssohn - NOAA Federal via R-package-devel wrote: > -- Forwarded message -- > From: Roy Mendelssohn - NOAA Federal > To: "Maëlle SALMON via R-package-devel" > Cc: > Bcc: > Date: Tue, 16 Apr 2019 17:41:21 -0700 > Subject: Vignette timing out on test server > Hi All: > > The package I am trying to get ready for submission has a Vignette that does > a number of data downloads and some complicate maps with satellite images. > While it knits on my computer, on checks it times out. I am looking for > ways to speed it up, but my question is, is there a command or a way to > profile a vignette as it knits to have a record to see which chunks are > using up how much time? > > I realize CRAN only runs the vignette code to check that it runs, but it must > have a time limit because I am consistently hitting time outs on test > machines. So ti would help me if I can identify where all the time is. There you go: https://stackoverflow.com/questions/24595280/timing-for-chunks https://stackoverflow.com/questions/30530008/hook-to-time-knitr-chunks -- Iñaki Úcar __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Submitting a package whose unit tests sometimes fail because of server connections
5.4 In the spirit of simple & stupid you can also use the built in mechanism for doing this: organize some of your tests in subdirectories like inst/testWithInternet, inst/veryLongTests, inst/testsNeedingLicence, inst/testsNeedingSpecialCluster, etc. CRAN will only run the tests in the tests/ directory, but you can check them yourself using R CMD check --test-dir=inst/testWithInternet whatever.tar.gz > In a separate response On 4/16/19 2:06 PM, Steven Scott wrote: > Just don't include the live fire stuff in the package. Please do not do this. If you omit tests from your package then it cannot be properly checked by other people. Paul Gilbert On 4/16/19 2:16 PM, Dirk Eddelbuettel wrote: On 16 April 2019 at 11:40, Will wrote: | Some things I have considered include: | |1. Skipping all unit tests on CRAN (using something like |*testtht::skip_on_cran*). This would immediately fix the problem, and as |a mitigating factor we report automated test result and coverage on the |package's GitHub page (https://github.com/ropensci/suppdata). |2. Using HTTP-mocking to avoid requiring a call to a server during tests |at all. I would be uncomfortable relying solely on this for all tests, |since if the data hosters changed things we wouldn't know. Thus I would |still want the Internet-enabled tests, which would also have to be turned |off for CRAN (see 1 above). This would also be a lot of additional work. |3. Somehow bypassing the requirement for the unit tests to all pass |before the package is checked by the CRAN maintainers. I have no idea if |this is something CRAN would be willing to do, or if it is even possible. |It would be the easiest option for me, but I don't want to create extra |work for other people! |4. Slowing the tests with something like *Sys.sleep*. This might work, |but would slow the tests massively and so might that cause problems for |CRAN? | | Does anyone have any advice as to which of the above would be the best | option, or who I should email directly about this? 5. Run a hybrid scheme where you have multiple levels: 5.1 Do what eg Rcpp does and only opt into 'all tests' when an overall variable is set; that variable can be set conveniently in .travis.yml and conditionally in your test runner below ~/tests/ That way you can skip tests that would fail. 5.2 Do a lot of work and wrap 3. above into try() / tryCatch() and pass if _your own aggregation of tests_ passes a threshold. Overkill to me. 5.3 Turn all tests on / off based on some other toggle. I.e. I don't think I test all features of RcppRedis on CRAN as I can't assume a redis server, but I do run those tests at home, on Travis, ... Overall, I would recommend to 'keep it simple & stupid' (KISS) as life is to short to argue^Hdebate this with CRAN. And their time is too precious so we should try to make their life easier. Dirk __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Vignette timing out on test server
--- Begin Message --- Thanks muchly. That looks like I what I was after. otherwise I was going to have to put tictoc on each chunk. -Roy > On Apr 17, 2019, at 2:21 AM, Iñaki Ucar wrote: > > On Wed, 17 Apr 2019 at 02:41, Roy Mendelssohn - NOAA Federal via > R-package-devel wrote: >> -- Forwarded message -- >> From: Roy Mendelssohn - NOAA Federal >> To: "Maëlle SALMON via R-package-devel" >> Cc: >> Bcc: >> Date: Tue, 16 Apr 2019 17:41:21 -0700 >> Subject: Vignette timing out on test server >> Hi All: >> >> The package I am trying to get ready for submission has a Vignette that does >> a number of data downloads and some complicate maps with satellite images. >> While it knits on my computer, on checks it times out. I am looking for >> ways to speed it up, but my question is, is there a command or a way to >> profile a vignette as it knits to have a record to see which chunks are >> using up how much time? >> >> I realize CRAN only runs the vignette code to check that it runs, but it >> must have a time limit because I am consistently hitting time outs on test >> machines. So ti would help me if I can identify where all the time is. > > There you go: > > https://stackoverflow.com/questions/24595280/timing-for-chunks > https://stackoverflow.com/questions/30530008/hook-to-time-knitr-chunks > > -- > Iñaki Úcar ** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center ***Note new street address*** 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. --- End Message --- __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] dtrti2 error on CRAN checks
Dear Sirs, Since the start of April, I have gotten some errors on the Debian machines on the check on CRAN for my package dynamichazard. The places where I get the errors seems to come down to LAPACK's dtrtri routine. I have tried to reproduce the error on Debian 18.04.2 with clang 8.0.0 but I cannot reproduce them. The errors came suddenly and the tests passed before. I see similar issues (I think) in the following package: - RcppHMM - BNPmix - themetagenomics - tidytext - stm Any ideas what the error may be? Here is the relevant code parts for one of the failed tests: https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/for_tests.cpp#L216 https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/arma_BLAS_LAPACK.cpp#L26 https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/R_BLAS_LAPACK.cpp#L67 Sincerely yours, Benjamin __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] dtrti2 error on CRAN checks
On 17 April 2019 at 11:06, Benjamin Christoffersen wrote: | Since the start of April, I have gotten some errors on the Debian | machines on the check on CRAN for my package dynamichazard. The places | where I get the errors seems to come down to LAPACK's dtrtri routine. | I have tried to reproduce the error on Debian 18.04.2 with clang 8.0.0 | but I cannot reproduce them. | | The errors came suddenly and the tests passed before. I see similar | issues (I think) in the following package: | - RcppHMM | - BNPmix | - themetagenomics | - tidytext | - stm | | Any ideas what the error may be? Here is the relevant code parts for | one of the failed tests: | https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/for_tests.cpp#L216 | https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/arma_BLAS_LAPACK.cpp#L26 | https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/R_BLAS_LAPACK.cpp#L67 There are four or more different sources of LAPACK and BLAS on Debian as we (since the late 1990s !!) utilize the nature of the _interface_ allowing different packages to fill in. So there could be i) the R internal source with a partial library -- Fedora/CentOS use this but the Debian distro builds do not ii) "reference BLAS", external, unoptimized iii) OpenBLAS, the successor to Goto, parallel via multithreading iv) Atlas, "tuned" but not mulitthreading and more as eg Intel MKL via a quick script (see my blog). As a historical aside, and way-back-when, ii) earned us year's long growling from the direction of Oxfordshire because some early/old versions of LAPACK had bugs. But it cuts both ways: the external versions tend to be complete whereas what R ships with internally contains a subset -- and at least one (early) user of RcppArmadillo was bitten when R built from sources using defaults would not have the complex operations he needed. To R Core's credit these missing functions got filled in over the years. In short, you need to look more closely. On the Ubuntu 18.10 machine that I type this on, sessionInfo()'s second paragraph has Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.3.so which IIRC is also the default (via some Debian/Ubuntu internal 'ordering' of the available alternatives). Hth, Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] dtrti2 error on CRAN checks
Thanks for the quick reply. First a correction. I meant Ubuntu 18.04.2 of course, sorry. > In short, you need to look more closely. Is there a way to tell which BLAS and LAPACK is used on the Debian machines on CRAN? I checked the "CRAN Package Check Flavors" page but there is no information there regarding BLAS or LAPACK. I have tried to test the package with Atlas, OpenBlas, and the reference implementation with gcc 7.3.0 and 8.2.0. It all worked without any errors. It seems that gcc has changed since January from version 8.2.0 and 7.3.0 to 8.3.0 on the tests on CRAN. Further, it is now Debian version 8.3.0-2 instead of Debian 8.2.0-7 and Debian 7.3.0-29. I do not know whether this can matter. Sincerely yours, Benjamin Christoffersen Den ons. 17. apr. 2019 kl. 11.38 skrev Dirk Eddelbuettel : > > > On 17 April 2019 at 11:06, Benjamin Christoffersen wrote: > | Since the start of April, I have gotten some errors on the Debian > | machines on the check on CRAN for my package dynamichazard. The places > | where I get the errors seems to come down to LAPACK's dtrtri routine. > | I have tried to reproduce the error on Debian 18.04.2 with clang 8.0.0 > | but I cannot reproduce them. > | > | The errors came suddenly and the tests passed before. I see similar > | issues (I think) in the following package: > | - RcppHMM > | - BNPmix > | - themetagenomics > | - tidytext > | - stm > | > | Any ideas what the error may be? Here is the relevant code parts for > | one of the failed tests: > | > https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/for_tests.cpp#L216 > | > https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/arma_BLAS_LAPACK.cpp#L26 > | > https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/R_BLAS_LAPACK.cpp#L67 > > There are four or more different sources of LAPACK and BLAS on Debian as we > (since the late 1990s !!) utilize the nature of the _interface_ allowing > different packages to fill in. > > So there could be > i) the R internal source with a partial library -- Fedora/CentOS use this >but the Debian distro builds do not > ii) "reference BLAS", external, unoptimized > iii) OpenBLAS, the successor to Goto, parallel via multithreading > iv) Atlas, "tuned" but not mulitthreading > and more as eg Intel MKL via a quick script (see my blog). > > As a historical aside, and way-back-when, ii) earned us year's long growling > from the direction of Oxfordshire because some early/old versions of LAPACK > had bugs. But it cuts both ways: the external versions tend to be complete > whereas what R ships with internally contains a subset -- and at least one > (early) user of RcppArmadillo was bitten when R built from sources using > defaults would not have the complex operations he needed. To R Core's credit > these missing functions got filled in over the years. > > In short, you need to look more closely. On the Ubuntu 18.10 machine that I > type this on, sessionInfo()'s second paragraph has > > Matrix products: default > BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3 > LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.3.so > > which IIRC is also the default (via some Debian/Ubuntu internal 'ordering' of > the available alternatives). > > Hth, Dirk > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
