Re: [R-pkg-devel] suggestion: conda for third-party software
Le 08/01/2020 à 08:50, Ivan Krylov a écrit : On Tue, 7 Jan 2020 15:49:45 +0100 Serguei Sokol wrote: Currently, many R packages include TPS as part of them thus bloating their sizes and often duplicating files on a given system. And even when TPS is not included in an R package but is just installed on a system, it is not so obvious to get the right path to it. Sometimes pkg-config helps but it is not always present. I agree that making a package depend on a third-party library means finding oneself in a bit of a pickle. A really popular library like cURL could be "just" depended upon (for the price of some problems when building on Windows). A really small (e.g. 3 source files) and rarely updated (just once last year) library like liborigin could "just" be bundled (but the package maintainer would have to constantly watch out for new versions of the library). Finding that the bundled version of a network-facing library in an R package (e.g. libuv in httpuv) is several minor versions out of date is always a bit scary, even if it turns out that no major security flaws have been found in that version (just a few low-probability resource leaks, one unlikely NULL pointer dereference and some portability problems). The road to dependency hell is paved with intentions of code reuse. So, the new feature would be to let R package developers to write in DESCRIPTION/SystemRequirements field something like 'conda:boost-cpp>=1.71' where 'boost-cpp' is an example of a conda package and '>=1.71' is an optional version requirement. While I appreciate the effort behind Anaconda, I would hate to see it being *required* to depend on third-party binaries compiled by a fourth-party (am I counting my parties right?) when there's already a copy installed and available via means the user trusts more (e.g. via GNU/Linux distro package, or Homebrew on macOS, or just a copy sitting in /usr/local installed manually from source). In this regard, a separate field like "Config/conda" suggested by Kevin Ushey sounds like a good idea: if one wants to use Anaconda, the field is there. If one doesn't, one can just ignore it and provide the necessary dependencies in a different way. The same would apply for my proposition: if you want, you use conda:something if not you do like before. But anyway, I don't make a campaign for 'conda:' tag in SystemRequirements. Kevin's Config/conda solution seems to be sufficient for this issue. Just, I was not aware that it was already there. Best, Serguei. __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Significant warnings during package checks
Hi R-package-devel team, I submitted my package on CRAN and below errors and warnings seemed to have appeared in the during the automatic check but that did not actually appear when I compile. Any suggestions on how this problem could be resolved? * checking whether package BASiNET can be installed … [6s/5s] WARNING Found the following significant warnings: Warning: 'rgl_init' failed, running with rgl.useNULL = TRUE * Warning in rgl.init(initValue, onlyNULL) : RGL: unable to open X11 display Warning: 'rgl_init' failed, running with rgl.useNULL = TRUE Thank you as I anticipate your kind response, MA [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Significant warnings during package checks
On 08/01/2020 7:48 a.m., MONSUUR OLAWALE via R-package-devel wrote: Hi R-package-devel team, I submitted my package on CRAN and below errors and warnings seemed to have appeared in the during the automatic check but that did not actually appear when I compile. Any suggestions on how this problem could be resolved? * checking whether package BASiNET can be installed … [6s/5s] WARNING Found the following significant warnings: Warning: 'rgl_init' failed, running with rgl.useNULL = TRUE * Warning in rgl.init(initValue, onlyNULL) : RGL: unable to open X11 display Warning: 'rgl_init' failed, running with rgl.useNULL = TRUE It appears that you are trying to open an rgl display. This will happen automatically if you load rgl without first specifying that you don't need a display. (You do that by setting option(rgl.useNULL = TRUE). What sort of dependency on rgl does your package have? Duncan Murdoch __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] suggestion: conda for third-party software
On Tue, Jan 7, 2020 at 10:42 PM Sokol Serguei wrote: > > Thanks for this hint. > > Le 07/01/2020 à 20:47, Kevin Ushey a écrit : > > The newest version of reticulate does something very similar: R > > packages can declare their Python package dependencies in the > > Config/reticulate field of a DESCRIPTION file, and reticulate can read > > and use those dependencies to provision a Python environment for the > > user when requested (currently using Miniconda). > > If miniconda is used, does it mean that not only Python but any conda > package can be indicated in dependency ? In theory yes, but reticulate only accepts Python package dependencies since its primary goal is interoperation with Python. > And another question, do you know if miniconda is installed on testing > CRAN machines? (Without this I cannot see how your packages with conda > dependencies could be tested during their submission.) I don't think so. I can't speak for CRAN, but their time is precious and it seems unlikely to me that they would be willing to expend the time needed to maintain Conda installations across their fleet of CRAN machines. Packages using Miniconda in this way could still run their tests on different types of infrastructure, though (e.g. Travis CI). > Best, > > Serguei. > > > > > Similarly, rather than having this part of SystemRequirements, package > > authors could declare these in a separate field called e.g. > > Config/conda. Then, you could have an R package that knows how to read > > and parse these configuration requests, and install those packages for > > the user. > > > > That said, maintaining a Conda installation and its environments is > > non-trivial, and things do not always work as expected when mixing > > Conda applications with non-Conda applications. Most notably, Conda > > installations bundle their own copies of libraries; e.g. the C++ > > standard library, Qt, OpenSSL, and so on. If an application tries to > > mix and match both system-provided and Conda-provided libraries in the > > same process, bad things often happen. This was still the > > lowest-friction way forward for us with reticulate, but it's worth > > being aware that Conda is not a total panacea. > > > > Best, > > Kevin > > > > On Tue, Jan 7, 2020 at 6:50 AM Serguei Sokol > > wrote: > >> Best wishes for 2020! > >> > >> I would like to suggest a new feature for R package management. Its aim > >> is to enable package developers and end-users to rely on conda ( > >> https://docs.conda.io/en/latest/ ) for managing third-party software > >> (TPS) on major platforms: linux64, win64 and osx64. Currently, many R > >> packages include TPS as part of them thus bloating their sizes and often > >> duplicating files on a given system. And even when TPS is not included > >> in an R package but is just installed on a system, it is not so obvious > >> to get the right path to it. Sometimes pkg-config helps but it is not > >> always present. > >> > >> So, the new feature would be to let R package developers to write in > >> DESCRIPTION/SystemRequirements field something like > >> 'conda:boost-cpp>=1.71' where 'boost-cpp' is an example of a conda > >> package and '>=1.71' is an optional version requirement. Having this > >> could allow install.packages() to install TPS on a testing CRAN machine > >> or on an end-user's one. (There is just one line to execute in a shell: > >> conda install . It will install the package itself as well as > >> all its dependencies). > >> > >> To my mind, this feature would have the following advantages: > >>- on-disk size economy as the same TPS does not have to be included in > >> R package itself and can be shared with other language wrappers, e.g. > >> Python; > >>- an easy flag configuring in Makevars as paths to TPS will be well > >> known in advance; > >>- CRAN machines could test packages relying on a wide panel of TPS > >> without bothering with their manual installation; > >>- TPS installation can become transparent for the end-user on major > >> platforms; > >> > >> Note that even R is part of conda ( > >> https://anaconda.org/conda-forge/r-base ), it is not mandatory to use > >> the conda's R version for this feature. Here, conda is just meant to > >> facilitate access to TPS. However, a minimal requirement is obviously to > >> have conda itself. > >> > >> Does it look reasonable? appealing? > >> Best, > >> Serguei. > >> > >> __ > >> R-package-devel@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-package-devel > > __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] pandoc lua filter error with win devel check
Hi, I added SystemRequirements: pandoc (>= 2.0) to DESCRIPTION as you suggested; submission failed, the automatic check of package submission to CRAN had the error I got before with check_win_devel: "pandoc.exe: unrecognized option `--lua-filter=refs/multiple-bibliographies.lua' Error: processing vignette 'index.Rmd' failed with diagnostics: pandoc document conversion failed with error 2" So, I am not sure what would be the next step for submitting. Related info: Vignette failure because pandoc ver. < 2 in Windows-R-Devel. Pandoc (>2) needed for multiple bibliographies in vignette (with lua filter), see also: https://stat.ethz.ch/pipermail/r-package-devel/2019q2/004127.html https://stat.ethz.ch/pipermail/r-package-devel/2020q1/004814.html https://cran.r-project.org/web/packages/rmarkdown/vignettes/lua-filters.html https://pandoc.org/lua-filters.html https://github.com/jgm/pandoc/releases (pandoc builds for every OS) Best, On Mon, Jan 6, 2020 at 8:03 PM Max Turgeon wrote: > There was a discussion on this list last year about which version of > pandoc is available on CRAN machines, without a definite answer: > https://stat.ethz.ch/pipermail/r-package-devel/2019q2/004126.html > > > But I think it's completely possible that win-builder doesn't have pandoc > 2.0, which is required for lua-filters AFAIK. On your side of things, the > best you can do is to ensure that you add > "SystemRequirements: pandoc (>= 2.0)" (or something like that) to your > DESCRIPTION file, and add a corresponding justification to your submission > comments. > > > Max Turgeon > Assistant Professor > Department of Statistics > Department of Computer Science > University of Manitoba > maxturgeon.ca > > > -- > *From:* R-package-devel on behalf > of Fernando Roa > *Sent:* January 6, 2020 4:26:14 PM > *To:* Jeff Newmiller; r-package-devel@r-project.org > *Subject:* Re: [R-pkg-devel] pandoc lua filter error with win devel check > > Hi > Rmarkdown supports lua filters. So I guess the rmarkdown and or pandoc in > win devel are not updated. > > Best, > > > On Mon, Jan 6, 2020, 7:18 PM Jeff Newmiller > wrote: > > > Pandoc is not part of the standard R documentation tooling... you must be > > doing something special, and doing something special makes getting a > > package accepted more challenging. Perhaps if you were more specific you > > might get a more specific response. > > > > On January 6, 2020 1:38:44 PM PST, Fernando Roa > wrote: > > >Hi people, > > > > > >I got an error in vignettes after checking as win. devel. with > > >devtools (check_win_devel): > > > > > >pandoc.exe: unrecognized option > > >`--lua-filter=refs/multiple-bibliographies.lua' > > > > > >May I ignore it? > > > > > >Best, > > > > -- > > Sent from my phone. Please excuse my brevity. > > > > [[alternative HTML version deleted]] > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel > -- Fernando Roa [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] debian-clang error with class
Hi there,
I have an ERROR in my fpc package on debian-clang that I currently
cannot reproduce due to the fact that rhub is apparently affected by a
bug on that platform that doesn't allow me to install Bioconductor
packages. The description is "--- failure: the condition has length > 1
---". I have googled for this and have found that there are a lot of
packages that have such an error on debian-clang only. In all cases this
seems to be associated with lines that compare a class of some object
with a string such as
if (class(xxx)!="try-error"), if (class(m)=="matrix"), if (class(y) %in%
c("onething,"anotherthing")) etc.
(The fpc package apparently achieved this with a "try-error" check.)
Now on my machine (which isn't debian-clang) the class of xxx is a
harmless thing, either "try-error" or another simple string (in the
example that caused the error it is actually "matrix"), so it's not
clear to me what's wrong with this on debian-clang. However, looking at
all these other cases in other packages, I'm pretty sure that the error
comes from comparing class(something) with a string.
Can anybody shed some light on this? What goes wrong with
class(xxx)!="try-error" and the like on debian-clang that works elsewhere?
Thanks,
Christian
--
Christian Hennig
Dipartimento di Scienze Statistiche "Paolo Fortunati",
Universita di Bologna, phone +39 05120 98163
christian.hen...@unibo.it
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] debian-clang error with class
Take a look at
https://stat.ethz.ch/pipermail/r-package-devel/2019q4/004675.html
Best,
On Wed, Jan 8, 2020 at 3:09 PM Christian Martin Hennig <
christian.hen...@unibo.it> wrote:
> Hi there,
>
> I have an ERROR in my fpc package on debian-clang that I currently
> cannot reproduce due to the fact that rhub is apparently affected by a
> bug on that platform that doesn't allow me to install Bioconductor
> packages. The description is "--- failure: the condition has length > 1
> ---". I have googled for this and have found that there are a lot of
> packages that have such an error on debian-clang only. In all cases this
> seems to be associated with lines that compare a class of some object
> with a string such as
>
> if (class(xxx)!="try-error"), if (class(m)=="matrix"), if (class(y) %in%
> c("onething,"anotherthing")) etc.
>
> (The fpc package apparently achieved this with a "try-error" check.)
>
> Now on my machine (which isn't debian-clang) the class of xxx is a
> harmless thing, either "try-error" or another simple string (in the
> example that caused the error it is actually "matrix"), so it's not
> clear to me what's wrong with this on debian-clang. However, looking at
> all these other cases in other packages, I'm pretty sure that the error
> comes from comparing class(something) with a string.
>
> Can anybody shed some light on this? What goes wrong with
> class(xxx)!="try-error" and the like on debian-clang that works elsewhere?
>
> Thanks,
>
> Christian
>
>
> --
> Christian Hennig
> Dipartimento di Scienze Statistiche "Paolo Fortunati",
> Universita di Bologna, phone +39 05120 98163
> christian.hen...@unibo.it
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>
--
Fernando Roa
[[alternative HTML version deleted]]
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] debian-clang error with class
Great, thanks!
On 1/8/20 7:18 PM, Fernando Roa wrote:
Take a look at
https://stat.ethz.ch/pipermail/r-package-devel/2019q4/004675.html
Best,
On Wed, Jan 8, 2020 at 3:09 PM Christian Martin Hennig
mailto:christian.hen...@unibo.it>> wrote:
Hi there,
I have an ERROR in my fpc package on debian-clang that I currently
cannot reproduce due to the fact that rhub is apparently affected by a
bug on that platform that doesn't allow me to install Bioconductor
packages. The description is "--- failure: the condition has length > 1
---". I have googled for this and have found that there are a lot of
packages that have such an error on debian-clang only. In all cases this
seems to be associated with lines that compare a class of some object
with a string such as
if (class(xxx)!="try-error"), if (class(m)=="matrix"), if (class(y) %in%
c("onething,"anotherthing")) etc.
(The fpc package apparently achieved this with a "try-error" check.)
Now on my machine (which isn't debian-clang) the class of xxx is a
harmless thing, either "try-error" or another simple string (in the
example that caused the error it is actually "matrix"), so it's not
clear to me what's wrong with this on debian-clang. However, looking at
all these other cases in other packages, I'm pretty sure that the error
comes from comparing class(something) with a string.
Can anybody shed some light on this? What goes wrong with
class(xxx)!="try-error" and the like on debian-clang that works elsewhere?
Thanks,
Christian
--
Christian Hennig
Dipartimento di Scienze Statistiche "Paolo Fortunati",
Universita di Bologna, phone +39 05120 98163
christian.hen...@unibo.it
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Fernando Roa
--
Christian Hennig
Dipartimento di Scienze Statistiche "Paolo Fortunati",
Universita di Bologna, phone +39 05120 98163
christian.hen...@unibo.it
[[alternative HTML version deleted]]
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] suggestion: conda for third-party software
It would also be worth looking at the basilisk package: https://github.com/LTLA/basilisk where the approach used there is to instead embed a Conda installation as part of the R package itself. This comes with the benefit that it's now the package author's responsibility to maintain the Conda installation (not CRAN nor the users), but does have the drawback that installing or upgrading that Conda environment may become more challenging. One other large benefit of this approach is that it forces R package authors who want to use Python through reticulate to standardize on the same environment. Note that reticulate can only bind to a single Python session per R session, so attempting to have R packages which use incompatible Python dependencies could quickly become an issue. (Python packages tend to rely on virtual environments, and so Python packages tend to declare more narrow dependency version requirements.) Hence, having a "standardized" Python environment that can be used by R packages through reticulate (or other Python-wrapping packages) should be very useful. If you're curious, there's a more detailed discussion here: https://github.com/LTLA/basilisk/issues/2 Best, Kevin On Wed, Jan 8, 2020 at 8:34 AM Kevin Ushey wrote: > > On Tue, Jan 7, 2020 at 10:42 PM Sokol Serguei wrote: > > > > Thanks for this hint. > > > > Le 07/01/2020 à 20:47, Kevin Ushey a écrit : > > > The newest version of reticulate does something very similar: R > > > packages can declare their Python package dependencies in the > > > Config/reticulate field of a DESCRIPTION file, and reticulate can read > > > and use those dependencies to provision a Python environment for the > > > user when requested (currently using Miniconda). > > > > If miniconda is used, does it mean that not only Python but any conda > > package can be indicated in dependency ? > > In theory yes, but reticulate only accepts Python package dependencies > since its primary goal is interoperation with Python. > > > And another question, do you know if miniconda is installed on testing > > CRAN machines? (Without this I cannot see how your packages with conda > > dependencies could be tested during their submission.) > > I don't think so. I can't speak for CRAN, but their time is precious > and it seems unlikely to me that they would be willing to expend the > time needed to maintain Conda installations across their fleet of CRAN > machines. > > Packages using Miniconda in this way could still run their tests on > different types of infrastructure, though (e.g. Travis CI). > > > Best, > > > > Serguei. > > > > > > > > Similarly, rather than having this part of SystemRequirements, package > > > authors could declare these in a separate field called e.g. > > > Config/conda. Then, you could have an R package that knows how to read > > > and parse these configuration requests, and install those packages for > > > the user. > > > > > > That said, maintaining a Conda installation and its environments is > > > non-trivial, and things do not always work as expected when mixing > > > Conda applications with non-Conda applications. Most notably, Conda > > > installations bundle their own copies of libraries; e.g. the C++ > > > standard library, Qt, OpenSSL, and so on. If an application tries to > > > mix and match both system-provided and Conda-provided libraries in the > > > same process, bad things often happen. This was still the > > > lowest-friction way forward for us with reticulate, but it's worth > > > being aware that Conda is not a total panacea. > > > > > > Best, > > > Kevin > > > > > > On Tue, Jan 7, 2020 at 6:50 AM Serguei Sokol > > > wrote: > > >> Best wishes for 2020! > > >> > > >> I would like to suggest a new feature for R package management. Its aim > > >> is to enable package developers and end-users to rely on conda ( > > >> https://docs.conda.io/en/latest/ ) for managing third-party software > > >> (TPS) on major platforms: linux64, win64 and osx64. Currently, many R > > >> packages include TPS as part of them thus bloating their sizes and often > > >> duplicating files on a given system. And even when TPS is not included > > >> in an R package but is just installed on a system, it is not so obvious > > >> to get the right path to it. Sometimes pkg-config helps but it is not > > >> always present. > > >> > > >> So, the new feature would be to let R package developers to write in > > >> DESCRIPTION/SystemRequirements field something like > > >> 'conda:boost-cpp>=1.71' where 'boost-cpp' is an example of a conda > > >> package and '>=1.71' is an optional version requirement. Having this > > >> could allow install.packages() to install TPS on a testing CRAN machine > > >> or on an end-user's one. (There is just one line to execute in a shell: > > >> conda install . It will install the package itself as well as > > >> all its dependencies). > > >> > > >> To my mind, this feature would have the following advantages: > > >>- on-disk size economy a
[R-pkg-devel] Checkpoint package failing CRAN checks
Hi all, I'm the new (as of 6 months ago) maintainer of checkpoint. If you're not familiar with it, checkpoint is a package to help with reproducible analysis: you give it a particular snapshot date from the MRAN site, and it installs the packages that your project depends on from that snapshot. Just before the Christmas break, I got a message that it was failing CRAN checks. Specifically, building a vignette was causing a warning that a file was not found. I've made some changes, but the error persists. This is the snippet of the log with the error. The full log can be found at https://win-builder.r-project.org/incoming_pretest/checkpoint_0.4.8_20200108_065428/Debian/00check.log Error(s) in re-building vignettes: ... --- re-building 'checkpoint.Rmd' using rmarkdown * installing *source* package 'darts' ... ** using staged installation ** libs gcc-9 -I"/home/hornik/tmp/R-d-gcc-9/include" -DNDEBUG -I/usr/local/include -DUSE_TYPE_CHECKING_STRICT -D_FORTIFY_SOURCE=2 -fpic -g -O2 -Wall -pedantic -mtune=native -c darts.c -o darts.o gcc-9 -shared -L/home/hornik/tmp/R-d-gcc-9/lib -Wl,-O1 -o darts.so darts.o -L/home/hornik/tmp/R-d-gcc-9/lib -lR installing to /tmp/RtmpNSFOaW/working_dir/RtmpshQt0J/.checkpoint/2017-04-01/lib/x86_64-pc-linux-gnu/4.0.0/00LOCK-darts/00new/darts/libs ** R ** byte-compile and prepare package for lazy loading ** help *** installing help indices ** building package indices ** testing if installed package can be loaded from temporary location ** checking absolute paths in shared objects and dynamic libraries ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (darts) Warning in file(con, "w") : cannot open file '/tmp/RtmpNSFOaW/working_dir/RtmpshQt0J/rmarkdown-str2e0253dd0590.html': No such file or directory Error: processing vignette 'checkpoint.Rmd' failed with diagnostics: cannot open the connection --- failed re-building 'checkpoint.Rmd' Would anyone be able to provide a clue as to what's going on? It seems to be working on Windows, but not on Debian. The full package source is at https://github.com/RevolutionAnalytics/checkpoint. Thanks! __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
