[Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
Dear R-devel people, We are working to submit a package which is mainly a binding over a C++ lib (https://github.com/libKriging) using armadillo. It is _not_ a standard RcppArmadillo package, because we also had to provide a python binding... so there is a huge layer of cmake & scripting to make it work with a standard armadillo (but using same version that RcppArmadillo). It seems now working with almost all CRAN targets, but a problem remains with fedora (clang & gcc) devel. Our issue is that the rhub docker is not well sync with the CRAN fedora-clang-devel config: - failing on CRAN (without clear reason): https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-fedora-clang/rlibkriging-00check.html - passing on rhub: https://builder.r-hub.io/status/rlibkriging_0.7-3.tar.gz-20f7dc7562726497af7c678ab41f4dea So we cannot investigate and fix the problem. Note that we did quite strange things with the fedora platforms: include explicitely slapack to provide simple precision for our (vanilla) armadillo... Do you have any idea, or even known problem in mind, that could be related to this ? Best regards, -- Dr. Yann Richet Institute for Radiological Protection and Nuclear Safety (https://www.irsn.fr), Department of Characterization of Natural Unexpected Events and Sites Office : +33 1 58 35 88 84 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
Thank you for your answer. In facts, 10 threads are asked by armadillo for some LinAlg, which backs to two threads as warned. But I cannot imagine this costs so much time just for that... A deeper analysis of time spent seems to point that a large time was mainly spent on testthat and Rcpp dependencies compilation... But other recent packages depending on these also are not spending so much time. CRAN team, as I failed to reproduce the issue with rhub dockers, - is there any reason that could explain that fedora-*-devel is so slow for this package or compilation of Rcpp/testthat ? - is our slapack a possible source of... anything ? - are we the only package which embeds "standard armadillo" and tries to deal with simple precision lapack issues on fedora ? - is there any chance that I can get a deeper log of what happened ? Best regards, Yann -Message d'origine- De : Serguei Sokol Envoyé : mardi 10 janvier 2023 11:41 À : RICHET Yann ; R-devel@r-project.org Cc : Pascal Havé Objet : Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack Le 10/01/2023 à 11:37, Serguei Sokol a écrit : > Le 10/01/2023 à 10:44, RICHET Yann a écrit : >> Dear R-devel people, >> >> We are working to submit a package which is mainly a binding over a >> C++ lib (https://github.com/libKriging) using armadillo. >> It is _not_ a standard RcppArmadillo package, because we also had to >> provide a python binding... so there is a huge layer of cmake & >> scripting to make it work with a standard armadillo (but using same >> version that RcppArmadillo). >> It seems now working with almost all CRAN targets, but a problem >> remains with fedora (clang & gcc) devel. >> >> Our issue is that the rhub docker is not well sync with the CRAN >> fedora-clang-devel config: >> - failing on CRAN (without clear reason): >> https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-fedora-c >> lang/rlibkriging-00check.html > I see 2 candidates for reasons of failing on CRAN: > 1. test time is 30 min while 15 min are allowed; > 2. your code try to get 30 threads Oops, it was 10 threads not 30 but it does not change the nature of a possible issue. > while CRAN limits them to 2; > > Try to make your tests shorter ( < 15 min) on 2 threads. May be it > will help. > > Best, > Serguei. > >> - passing on rhub: >> https://builder.r-hub.io/status/rlibkriging_0.7-3.tar.gz-20f7dc756272 >> 6497af7c678ab41f4dea >> >> So we cannot investigate and fix the problem. >> >> Note that we did quite strange things with the fedora platforms: >> include explicitely slapack to provide simple precision for our >> (vanilla) armadillo... >> >> Do you have any idea, or even known problem in mind, that could be >> related to this ? >> >> Best regards, >> >> -- >> Dr. Yann Richet >> Institute for Radiological Protection and Nuclear Safety >> (https://www.irsn.fr), >> Department of Characterization of Natural Unexpected Events and >> Sites Office : +33 1 58 35 88 84 >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > -- Serguei Sokol Ingenieur de recherche INRAE Cellule Mathématiques TBI, INSA/INRAE UMR 792, INSA/CNRS UMR 5504 135 Avenue de Rangueil 31077 Toulouse Cedex 04 tel: +33 5 61 55 98 49 email: so...@insa-toulouse.fr http://www.toulouse-biotechnology-institute.fr/en/technology_platforms/mathematics-cell.html __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
Thank you all, for these advices.
So I try to fix OMP_THREADS, cleanup tests, and display explicitly what test is
running by moving in tests/ instead of tests/testthat/...
Next step should be to investigate blocking test using a reporter (maybe
"list").
For now, waiting for CRAN results...
Yann
-Message d'origine-
De : Duncan Murdoch
Envoyé : mercredi 11 janvier 2023 00:36
À : Sebastian Meyer ; Ivan Krylov ;
RICHET Yann
Cc : Pascal Havé ; R-devel@r-project.org
Objet : Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
On 10/01/2023 4:07 p.m., Sebastian Meyer wrote:
> Am 10.01.23 um 21:28 schrieb Duncan Murdoch:
>> On 10/01/2023 2:05 p.m., Ivan Krylov wrote:
>>> On Tue, 10 Jan 2023 16:27:53 +
>>> RICHET Yann wrote:
>>>
>>>> In facts, 10 threads are asked by armadillo for some LinAlg, which
>>>> backs to two threads as warned.
>>>
>>> I think you're right about your tests de-facto using two threads,
>>> but it might be a good idea to _default_ to up to two threads in
>>> tests and examples. This is especially valuable for third-party
>>> developers who have to mass-test packages (one of which could be
>>> rlibkriging) in parallel.
>>>
>>>> - is there any reason that could explain that fedora-*-devel is so
>>>> slow for this package or compilation of Rcpp/testthat ?
>>>
>>> Compilation time is definitely not the reason. Something in tests/*
>>> actually runs for 30 minutes by itself.
>>>
>>>> - is there any chance that I can get a deeper log of what happened ?
>>>
>>> If you split your tests into separate files under tests/*.R instead
>>> of using a single tests/testthat.R calling the rest of the tests, R
>>> will be able to show you the individual test file that hung and
>>> maybe the line where it happened. (Also, you'll get per-file
>>> timing.) But that is potentially a huge investment: you may have to
>>> rewrite the tests to work outside the testthat harness, and you'd
>>> also have to prepare another CRAN submission just to have those
>>> tests run. It's also against CRAN policy to knowingly submit a package with
>>> unfixed ERRORs.
>>>
>>> (Currently, R can only tell you that the tests hung in the
>>> test_check('rlibkriging') call in the tests/testthat.R, which isn't
>>> precise enough.)
>>
>> You can specify a different "reporter" in the test_check() call, and
>> it will print more useful information. I don't think there's a
>> perfect one, but
>>
>> test_check('rlibkriging', reporter = "progress")
>>
>> should at least show you the tests that finished running before the
>> timeout.
>
> I had similar problems with testthat and timeouts when mass-checking
> packages on patched R versions. My notes say
>
>> ## testthat's 'LocationReporter' does cat() after each expectation ##
>> so we can see results even if timeout is reached
>> options(testthat.default_check_reporter = c("Location", "Check"))
>
> The help("LocationReporter") says: "This reporter simply prints the
> location of every expectation and error. This is useful if you're
> trying to figure out the source of a segfault, or you want to figure
> out which code triggers a C/C++ breakpoint"
>
> HTH!
Yes, that looks like it would pin down the location of the problem.
Here's some sample output from it:
Running ‘testthat.R’ [41s/42s]
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
Start test: can use constructed calls in verify_output() (#945)
'test-verify-output.R:55' [success]
End test: can use constructed calls in verify_output() (#945)
Start test: verify_output() doesn't use cli unicode by default
'test-verify-output.R:65' [success]
'test-verify-output.R:73' [success]
End test: verify_output() doesn't use cli unicode by default
Start test: verify_output() handles carriage return
'test-verify-output.R:82' [success]
End test: verify_output() handles carriage return
Error: Test failures
Execution halted
One other thing: you enabled this by using
options(testthat.default_check_reporter = c("Location", "Check"))
before running the tests; the package writer could do the same thing by using
test_check('rlibkriging', reporter = c("Location", "Check"))
Duncan Murdoch
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
Well, I tried to move the tests outside testthat.R logic, because I expect that
CRAN output will not give me the reporter results... and as I re-submitted the
package, I wanted to ensure readable result. But maybe I am wrong about that...
?
-Message d'origine-
De : Duncan Murdoch
Envoyé : mercredi 11 janvier 2023 19:09
À : RICHET Yann ; Sebastian Meyer ; Ivan
Krylov
Cc : Pascal Havé ; R-devel@r-project.org
Objet : Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
On 11/01/2023 12:35 p.m., RICHET Yann wrote:
> Thank you all, for these advices.
>
> So I try to fix OMP_THREADS, cleanup tests, and display explicitly what test
> is running by moving in tests/ instead of tests/testthat/...
> Next step should be to investigate blocking test using a reporter (maybe
> "list").
> For now, waiting for CRAN results...
I think Sebastian or my suggestion is easier than redoing all of your tests.
They are each one line changes.
Duncan Murdoch
>
> Yann
>
> -Message d'origine-
> De : Duncan Murdoch Envoyé : mercredi 11
> janvier 2023 00:36 À : Sebastian Meyer ; Ivan Krylov
> ; RICHET Yann Cc : Pascal
> Havé ; R-devel@r-project.org Objet : Re: [Rd]
> rhub vs. CRAN fedora-*-devel, using armadillo & slapack
>
> On 10/01/2023 4:07 p.m., Sebastian Meyer wrote:
>> Am 10.01.23 um 21:28 schrieb Duncan Murdoch:
>>> On 10/01/2023 2:05 p.m., Ivan Krylov wrote:
>>>> On Tue, 10 Jan 2023 16:27:53 +
>>>> RICHET Yann wrote:
>>>>
>>>>> In facts, 10 threads are asked by armadillo for some LinAlg, which
>>>>> backs to two threads as warned.
>>>>
>>>> I think you're right about your tests de-facto using two threads,
>>>> but it might be a good idea to _default_ to up to two threads in
>>>> tests and examples. This is especially valuable for third-party
>>>> developers who have to mass-test packages (one of which could be
>>>> rlibkriging) in parallel.
>>>>
>>>>> - is there any reason that could explain that fedora-*-devel is so
>>>>> slow for this package or compilation of Rcpp/testthat ?
>>>>
>>>> Compilation time is definitely not the reason. Something in tests/*
>>>> actually runs for 30 minutes by itself.
>>>>
>>>>> - is there any chance that I can get a deeper log of what happened ?
>>>>
>>>> If you split your tests into separate files under tests/*.R instead
>>>> of using a single tests/testthat.R calling the rest of the tests, R
>>>> will be able to show you the individual test file that hung and
>>>> maybe the line where it happened. (Also, you'll get per-file
>>>> timing.) But that is potentially a huge investment: you may have to
>>>> rewrite the tests to work outside the testthat harness, and you'd
>>>> also have to prepare another CRAN submission just to have those
>>>> tests run. It's also against CRAN policy to knowingly submit a package
>>>> with unfixed ERRORs.
>>>>
>>>> (Currently, R can only tell you that the tests hung in the
>>>> test_check('rlibkriging') call in the tests/testthat.R, which isn't
>>>> precise enough.)
>>>
>>> You can specify a different "reporter" in the test_check() call, and
>>> it will print more useful information. I don't think there's a
>>> perfect one, but
>>>
>>> test_check('rlibkriging', reporter = "progress")
>>>
>>> should at least show you the tests that finished running before the
>>> timeout.
>>
>> I had similar problems with testthat and timeouts when mass-checking
>> packages on patched R versions. My notes say
>>
>>> ## testthat's 'LocationReporter' does cat() after each expectation
>>> ## so we can see results even if timeout is reached
>>> options(testthat.default_check_reporter = c("Location", "Check"))
>>
>> The help("LocationReporter") says: "This reporter simply prints the
>> location of every expectation and error. This is useful if you're
>> trying to figure out the source of a segfault, or you want to figure
>> out which code triggers a C/C++ breakpoint"
>>
>> HTH!
>
> Yes, that looks like it would pin down the location of the problem.
> Here's some sample output from it:
>
> Running ‘testthat.R’ [41s/42s]
> Running the tests in ‘tests/testthat.R’ failed.
&
Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
Thank you, Dirk. But I also tried with ccache, without fails... can you give some details about you reverse-depend configuration ? docker image ? Mine was a standard ubuntu 20.04 packages... Best, Yann -Message d'origine- De : Dirk Eddelbuettel Envoyé : mercredi 11 janvier 2023 19:35 À : RICHET Yann Cc : Duncan Murdoch ; Sebastian Meyer ; Ivan Krylov ; Pascal Havé ; R-devel@r-project.org Objet : Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack On 11 January 2023 at 17:35, RICHET Yann wrote: | Thank you all, for these advices. | | So I try to fix OMP_THREADS, cleanup tests, and display explicitly what test is running by moving in tests/ instead of tests/testthat/... | Next step should be to investigate blocking test using a reporter (maybe "list"). | For now, waiting for CRAN results... BTW your package failed its tests for me in a reverse-depends run when I had `ccache` as usual in the definition of CC, CXX, FC, CXX14, ... That usually works with other packages using CMake. Maybe something you could look into. Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack
The CRAN check on fedora gives me more detailed results now: https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-fedora-clang/rlibkriging-00check.html . It points a fairly strange issue (possibly due to recursive programming, that remains to investigate), but anyway I can now circumvent the problem for CRAN. In next "true" release, I will try to switch back to a testthat suitable reporter instead of the enumerated tests in tests/, as you suggested. So, thank you very much to all of you for your kind help and suggestions. Yann & libKriging team. -Message d'origine- De : Duncan Murdoch Envoyé : jeudi 12 janvier 2023 10:07 À : RICHET Yann ; Sebastian Meyer ; Ivan Krylov Cc : Pascal Havé ; R-devel@r-project.org Objet : Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack On 12/01/2023 2:51 a.m., RICHET Yann wrote: > Well, I tried to move the tests outside testthat.R logic, because I expect > that CRAN output will not give me the reporter results... and as I > re-submitted the package, I wanted to ensure readable result. But maybe I am > wrong about that... ? I think CRAN output will only show you the reporter results if there's an error. If it's a regular error, you get the last 13 lines. From your earlier posting, it looks as though a timeout might give more. But even the last 13 lines should identify exactly which test was running when the timeout happened. I probably wouldn't use the location reporter for routine runs, because it will give a lot of output, and conceivably this will slow things down. Duncan Murdoch > > > -Message d'origine- > De : Duncan Murdoch Envoyé : mercredi 11 > janvier 2023 19:09 À : RICHET Yann ; Sebastian > Meyer ; Ivan Krylov Cc : > Pascal Havé ; R-devel@r-project.org Objet : Re: > [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack > > On 11/01/2023 12:35 p.m., RICHET Yann wrote: >> Thank you all, for these advices. >> >> So I try to fix OMP_THREADS, cleanup tests, and display explicitly what test >> is running by moving in tests/ instead of tests/testthat/... >> Next step should be to investigate blocking test using a reporter (maybe >> "list"). >> For now, waiting for CRAN results... > > I think Sebastian or my suggestion is easier than redoing all of your tests. > They are each one line changes. > > Duncan Murdoch > >> >> Yann >> >> -Message d'origine- >> De : Duncan Murdoch Envoyé : mercredi 11 >> janvier 2023 00:36 À : Sebastian Meyer ; Ivan >> Krylov ; RICHET Yann Cc >> : Pascal Havé ; R-devel@r-project.org Objet : >> Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack >> >> On 10/01/2023 4:07 p.m., Sebastian Meyer wrote: >>> Am 10.01.23 um 21:28 schrieb Duncan Murdoch: >>>> On 10/01/2023 2:05 p.m., Ivan Krylov wrote: >>>>> On Tue, 10 Jan 2023 16:27:53 + RICHET Yann >>>>> wrote: >>>>> >>>>>> In facts, 10 threads are asked by armadillo for some LinAlg, >>>>>> which backs to two threads as warned. >>>>> >>>>> I think you're right about your tests de-facto using two threads, >>>>> but it might be a good idea to _default_ to up to two threads in >>>>> tests and examples. This is especially valuable for third-party >>>>> developers who have to mass-test packages (one of which could be >>>>> rlibkriging) in parallel. >>>>> >>>>>> - is there any reason that could explain that fedora-*-devel is >>>>>> so slow for this package or compilation of Rcpp/testthat ? >>>>> >>>>> Compilation time is definitely not the reason. Something in >>>>> tests/* actually runs for 30 minutes by itself. >>>>> >>>>>> - is there any chance that I can get a deeper log of what happened ? >>>>> >>>>> If you split your tests into separate files under tests/*.R >>>>> instead of using a single tests/testthat.R calling the rest of the >>>>> tests, R will be able to show you the individual test file that >>>>> hung and maybe the line where it happened. (Also, you'll get >>>>> per-file >>>>> timing.) But that is potentially a huge investment: you may have >>>>> to rewrite the tests to work outside the testthat harness, and >>>>> you'd also have to prepare another CRAN submission just to have >>>>> those tests run. It's also against CRAN policy to knowingly submit a
