Re: [R-pkg-devel] dtrti2 error on CRAN checks

[peter dalgaard]

Kurt Hornik submitted a bug report to the gfortran maintainers yesterday, and 
yes, it seems to have a lot to do with this.

He and Brian Ripley have been able to pinpoint it to certain gfortran revisions 
(specifically gfortran-9 at least r268992, or gfortran-8 at least r269349), but 
we're lacking a simple test case.

-pd

> On 18 Apr 2019, at 07:53 , Benjamin Christoffersen  wrote:
> 
> Thanks for the quick reply. First a correction. I meant Ubuntu 18.04.2
> of course, sorry.
> 
>> In short, you need to look more closely.
> Is there a way to tell which BLAS and LAPACK is used on the Debian
> machines on CRAN? I checked the "CRAN Package Check Flavors" page but
> there is no information there regarding BLAS or LAPACK.
> 
> I have tried to test the package with Atlas, OpenBlas, and the
> reference implementation with gcc 7.3.0 and 8.2.0. It all worked
> without any errors.
> 
> It seems that gcc has changed since January from version 8.2.0 and
> 7.3.0 to 8.3.0 on the tests on CRAN. Further, it is now Debian version
> 8.3.0-2 instead of Debian 8.2.0-7 and Debian 7.3.0-29. I do not know
> whether this can matter.
> 
> Sincerely yours,
> Benjamin Christoffersen
> 
> 
> 
> 
> 
> 
> 
> Den ons. 17. apr. 2019 kl. 11.38 skrev Dirk Eddelbuettel :
>> 
>> 
>> On 17 April 2019 at 11:06, Benjamin Christoffersen wrote:
>> | Since the start of April, I have gotten some errors on the Debian
>> | machines on the check on CRAN for my package dynamichazard. The places
>> | where I get the errors seems to come down to LAPACK's dtrtri routine.
>> | I have tried to reproduce the error on Debian 18.04.2 with clang 8.0.0
>> | but I cannot reproduce them.
>> |
>> | The errors came suddenly and the tests passed before. I see similar
>> | issues (I think) in the following package:
>> |  - RcppHMM
>> |  - BNPmix
>> |  - themetagenomics
>> |  - tidytext
>> |  - stm
>> |
>> | Any ideas what the error may be? Here is the relevant code parts for
>> | one of the failed tests:
>> | 
>> https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/for_tests.cpp#L216
>> | 
>> https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/arma_BLAS_LAPACK.cpp#L26
>> | 
>> https://github.com/boennecd/dynamichazard/blob/c42105feea5e2d16a8d461b7c606245a743b3e0a/src/BLAS_LAPACK/R_BLAS_LAPACK.cpp#L67
>> 
>> There are four or more different sources of LAPACK and BLAS on Debian as we
>> (since the late 1990s !!)  utilize the nature of the _interface_ allowing
>> different packages to fill in.
>> 
>> So there could be
>>  i)   the R internal source with a partial library -- Fedora/CentOS use this
>>   but the Debian distro builds do not
>>  ii)  "reference BLAS", external, unoptimized
>>  iii) OpenBLAS, the successor to Goto, parallel via multithreading
>>  iv)  Atlas, "tuned" but not mulitthreading
>> and more as eg Intel MKL via a quick script (see my blog).
>> 
>> As a historical aside, and way-back-when, ii) earned us year's long growling
>> from the direction of Oxfordshire because some early/old versions of LAPACK
>> had bugs.  But it cuts both ways: the external versions tend to be complete
>> whereas what R ships with internally contains a subset -- and at least one
>> (early) user of RcppArmadillo was bitten when R built from sources using
>> defaults would not have the complex operations he needed. To R Core's credit
>> these missing functions got filled in over the years.
>> 
>> In short, you need to look more closely. On the Ubuntu 18.10 machine that I
>> type this on, sessionInfo()'s second paragraph has
>> 
>>  Matrix products: default
>>  BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
>>  LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.3.so
>> 
>> which IIRC is also the default (via some Debian/Ubuntu internal 'ordering' of
>> the available alternatives).
>> 
>> Hth, Dirk
>> 
>> --
>> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
> 
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd@cbs.dk  Priv: pda...@gmail.com

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Submitting a package whose unit tests sometimes fail because of server connections

[Will]

Hello everyone,

Thank you very much for your help with this! These are some excellent
ideas; I think we will go with either the mocking approach or a variant of
Dirk's suggestion to use a test threshold.

Thanks again!

Will

---

Need a phylogeny? Try phyloGenerator: original
 or new version

Measuring phylogenetic structure? Try install.packages('pez')

Will Pearse 
Assistant Professor of Biology, Utah State University
Office: +1-435-797-0831; Room LSB-333
Skype: will.pearse


On Tue, 16 Apr 2019 at 12:20, Iñaki Ucar  wrote:

> On Tue, 16 Apr 2019 at 19:57, Will  wrote:
> >
> > Hello everyone,
> >
> > I'm sorry to bother you, but I would like some help getting a package (
> > *suppdata*; https://github.com/ropensci/suppdata) available through
> CRAN.
> > The package downloads supplementary materials from papers and data
> > repositories on the basis of their DOI, making reproducible analyses a
> > little easier to share and create.
> >
> > The package's unit tests involve downloading data, and when multiple
> > requests for the same resource are sent from a single machine (as seems
> to
> > be done by CRAN's testing servers) the data hosts will sometimes refuse
> the
> > connection and so the package's tests will fail. I emphasise that the
> tests
> > pass at Travis-CI (https://travis-ci.org/ropensci/suppdata) and OpenCPU
> (
> > https://ropensci.ocpu.io/suppdata/info); I am confident this is a
> > connection issue but I would be grateful to be shown I am wrong. I am not
> > sure how to solve this problem, and was hoping you might have some
> advice.
> > Some things I have considered include:
> >
> >
> >1. Skipping all unit tests on CRAN (using something like
> >*testtht::skip_on_cran*). This would immediately fix the problem, and
> as
> >a mitigating factor we report automated test result and coverage on
> the
> >package's GitHub page (https://github.com/ropensci/suppdata).
>
> This doesn't seem a good idea.
>
> >2. Using HTTP-mocking to avoid requiring a call to a server during
> tests
> >at all. I would be uncomfortable relying solely on this for all tests,
> >since if the data hosters changed things we wouldn't know. Thus I
> would
> >still want the Internet-enabled tests, which would also have to be
> turned
> >off for CRAN (see 1 above). This would also be a lot of additional
> work.
>
> What about mocking on CRAN and run Internet-enabled tests otherwise?
>
> >3. Somehow bypassing the requirement for the unit tests to all pass
> >before the package is checked by the CRAN maintainers. I have no idea
> if
> >this is something CRAN would be willing to do, or if it is even
> possible.
> >It would be the easiest option for me, but I don't want to create
> extra
> >work for other people!
>
> I think it's acceptable to skip a test if something is not available,
> but not the majority of them, for packages like this which basically
> is about downloading stuff from APIs. Mocking on CRAN, as said before,
> is the way to go here.
>
> >4. Slowing the tests with something like *Sys.sleep*. This might work,
> >but would slow the tests massively and so might that cause problems
> for
> >CRAN?
>
> This is not a good idea either.
>
> --
> Iñaki Úcar
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel