Gabriel, It is the most basic CI use case. One wants to install only dependencies only of the package, and run R CMD check on the package.
Unless you say that installing the package and then running R CMD check on that package is considered good practice. Then yes, functionality I am asking about is not needed. Somehow I never thought that this could be considered a good practice just by the fact that installation of the package could already impact environment in which check is taking place. Best, Jan On Fri, Oct 28, 2022 at 8:42 PM Gabriel Becker <[email protected]> wrote: > > Hi Jan, > > The reason, I suspect without speaking for R-core, is that by design you > should not be specifying package dependencies as additional packages to > install. install.packages already does this for you, as it did in the > construct of a repository code that I provided previously in the thread. You > should be *only* doing > > install.packages(<pkg in question>, repos = *) > > Then everything happens automatically via extremely well tested very mature > code. > > I (still) don't understand why you'd need to pass install.packages the vector > of dependencies yourself, as that is counter to install.packages' core design. > > Does that make sense? > > Best, > ~G > > On Fri, Oct 28, 2022 at 12:18 PM Jan Gorecki <[email protected]> wrote: >> >> Gabriel, >> >> I am trying to design generic solution that could be applied to >> arbitrary package. Therefore I went with the latter solution you >> proposed. >> If we wouldn't have to exclude base packages, then its a 3 liner >> >> file.copy("DESCRIPTION", file.path(tdir<-tempdir(), "PACKAGES")); >> db<-available.packages(paste0("file://", tdir)); >> utils::install.packages(tools::package_dependencies("pkgname", db, >> which="most")[[1L]]) >> >> As you noticed, we still have to filter out base packages. Otherwise >> it won't be a robust utility that can be used in CI. Therefore we have >> to add a call to tools:::.get_standard_package_names() which is an >> internal function (as of now). Not only complicating the call but also >> putting the functionality outside of safe use. >> >> Considering above, don't you agree that the following one liner could >> nicely address the problem? The problem that hundreds/thousands of >> packages are now addressing in their CI scripts by using a third party >> packages. >> >> utils::install.packages(packages.dcf("DESCRIPTION", which="most")) >> >> It is hard to me to understand why R members don't consider this basic >> functionality to be part of base R. Possibly they just don't need it >> themselves. Yet isn't this sufficient that hundreds/thousands of >> packages does need this functionality? >> >> Best regards, >> Jan >> >> On Mon, Oct 17, 2022 at 8:39 AM Jan Gorecki <[email protected]> wrote: >> > >> > Gabriel and Simon >> > >> > I completely agree with what you are saying. >> > The thing is that obtaining recursive deps, all/most whatever, is already >> > well supported in core R. What is missing is just this single >> > functionality I am requesting. >> > >> > If you will look into the branch you can see there is mirror.packages >> > function meant to mirror a slice of CRAN. It is doing exactly what you >> > described: package_dependencies; to obtain recursive deps, then download >> > all, etc. >> > I would love to have this function provided by core R as well, but we need >> > to start somewhere. >> > >> > There are other use cases as well. >> > For example CI, where one wants to install all/most dependencies and then >> > run R CMD check. Then we don't worry about recursive deps are they will be >> > resolved automatically. >> > I don't think it's reasonable to force users to use 3rd party packages to >> > handle such a common and simple use case. Otherwise one has to hard code >> > deps in CI script. Not robust at all. >> > >> > packages.dcf and repos.dcf makes all that way easier, and are solid base >> > for building customized orchestration like mirroring slice of CRAN. >> > >> > Best regards >> > Jan >> > >> > On Sun, Oct 16, 2022, 01:31 Simon Urbanek <[email protected]> >> > wrote: >> >> >> >> Jan, >> >> >> >> I think using a single DCF as input is not very practical and would not >> >> be useful in the context you describe (creating self contained repos) >> >> since they typically concern a list of packages, but essentially >> >> splitting out the part of install.packages() which determines which files >> >> will be pulled from where would be very useful as it would be trivial to >> >> use it to create repository (what we always do in corporate environments) >> >> instead of installing the packages. I suspect that install packages is >> >> already too complex so instead of adding a flag to install.packages one >> >> could move that functionality into a separate function - we all do that >> >> constantly for the sites we manage, so it would be certainly something >> >> worthwhile. >> >> >> >> Cheers, >> >> Simon >> >> >> >> >> >> > On Oct 15, 2022, at 7:14 PM, Jan Gorecki <[email protected]> wrote: >> >> > >> >> > Hi Gabriel, >> >> > >> >> > It's very nice usage you provided here. Maybe instead of adding new >> >> > function we could extend packages_depenedncies then? To accept file >> >> > path to >> >> > dsc file. >> >> > >> >> > What about repos.dcf? Maybe additional repositories could be an >> >> > attribute >> >> > attached to returned character vector. >> >> > >> >> > The use case is to, for a given package sources, obtain its >> >> > dependencies, >> >> > so one can use that for installing them/mirroring CRAN subset, or >> >> > whatever. >> >> > The later is especially important for a production environment where one >> >> > wants to have fixed version of packages, and mirroring relevant subset >> >> > of >> >> > CRAN is the most simple, and IMO reliable, way to manage such >> >> > environment. >> >> > >> >> > Regards >> >> > Jan >> >> > >> >> > On Fri, Oct 14, 2022, 23:34 Gabriel Becker <[email protected]> >> >> > wrote: >> >> > >> >> >> Hi Jan and Jan, >> >> >> >> >> >> Can you explain a little more what exactly you want the non-recursive, >> >> >> non-version aware dependencies from an individual package for? >> >> >> >> >> >> Either way package_dependencies will do this for you* with a little >> >> >> "aggressive convincing". It wants output from available.packages, but >> >> >> who >> >> >> really cares what it wants? It's a function and we are people :) >> >> >> >> >> >>> library(tools) >> >> >>> db <- read.dcf("~/gabe/checkedout/rtables_clean/DESCRIPTION") >> >> >>> package_dependencies("rtables", db, which = intersect(c("Depends", >> >> >> "Suggests", "Imports", "LinkingTo"), colnames(db))) >> >> >> $rtables >> >> >> [1] "methods" "magrittr" "formatters" "dplyr" "tibble" >> >> >> [6] "tidyr" "testthat" "xml2" "knitr" "rmarkdown" >> >> >> [11] "flextable" "officer" "stats" "htmltools" "grid" >> >> >> >> >> >> >> >> >> The only gotcha that I see immediately is that "LinkingTo" isn't always >> >> >> there (whereas it is with real output from available.packages). If you >> >> >> know your package doesn't have that (or that it does) at call time , >> >> >> this >> >> >> becomes a one-liner: >> >> >> >> >> >> package_dependencies("rtables", db = >> >> >> read.dcf("~/gabe/checkedout/rtables_clean/DESCRIPTION"), which = >> >> >> c("Depends", "Suggests", "Imports")) >> >> >> $rtables >> >> >> [1] "methods" "magrittr" "formatters" "dplyr" "tibble" >> >> >> [6] "tidyr" "testthat" "xml2" "knitr" "rmarkdown" >> >> >> [11] "flextable" "officer" "stats" "htmltools" "grid" >> >> >> >> >> >> You can also trick it a slightly different way by giving it what it >> >> >> actually wants >> >> >> >> >> >>> tdir <- tempdir() >> >> >>> file.copy("~/gabe/checkedout/rtables_clean/DESCRIPTION", >> >> >>> file.path(tdir, >> >> >> "PACKAGES")) >> >> >> [1] TRUE >> >> >>> avl <- available.packages(paste0("file://", tdir)) >> >> >>> library(tools) >> >> >>> package_dependencies("rtables", avl) >> >> >> $rtables >> >> >> [1] "methods" "magrittr" "formatters" "stats" "htmltools" >> >> >> [6] "grid" >> >> >> >> >> >>> package_dependencies("rtables", avl, which = "all") >> >> >> $rtables >> >> >> [1] "methods" "magrittr" "formatters" "stats" "htmltools" >> >> >> [6] "grid" "dplyr" "tibble" "tidyr" "testthat" >> >> >> [11] "xml2" "knitr" "rmarkdown" "flextable" "officer" >> >> >> >> >> >> So the only real benefits I see that we'd be picking up here is >> >> >> automatic >> >> >> filtering by priority, and automatic extraction of the package name >> >> >> from >> >> >> the DESCRIPTION file. I'm not sure either of those warrant a new >> >> >> exported >> >> >> function that R-core has to maintain forever. >> >> >> >> >> >> Best, >> >> >> ~G >> >> >> >> >> >> * I haven't tested this across all OSes, but I dont' know of any >> >> >> reason it >> >> >> wouldn't work generally. >> >> >> >> >> >> On Fri, Oct 14, 2022 at 2:33 PM Jan Gorecki <[email protected]> >> >> >> wrote: >> >> >> >> >> >>> Hello Jan, >> >> >>> >> >> >>> Thanks for confirming about many packages reinventing this missing >> >> >>> functionality. >> >> >>> packages.dcf was not meant handle versions. It just extracts names of >> >> >>> dependencies... Yes, such a simple thing, yet missing in base R. >> >> >>> >> >> >>> Versions of packages can be controlled when setting up R pkgs repo. >> >> >>> This >> >> >>> is >> >> >>> how I used to handle it. Making a CRAN subset mirror of fixed version >> >> >>> pkgs. >> >> >>> BTW. function for that is also included in mentioned branch. I am >> >> >>> just not >> >> >>> proposing it, to increase the chance of having at least this simple, >> >> >>> missing, functionality merged. >> >> >>> >> >> >>> Best >> >> >>> Jan >> >> >>> >> >> >>> On Fri, Oct 14, 2022, 15:14 Jan Netík <[email protected]> wrote: >> >> >>> >> >> >>>> Hello Jan, >> >> >>>> >> >> >>>> I have seen many packages that implemented dependencies "extraction" >> >> >>>> on >> >> >>>> their own for internal purposes and today I was doing exactly that >> >> >>>> for >> >> >>>> mine. It's not a big deal using read.dcf on DESCRIPTION. It was >> >> >>> sufficient >> >> >>>> for me, but I had to take care of some \n chars (the overall returned >> >> >>> value >> >> >>>> has some rough edges, in my opinion). However, the function from the >> >> >>> branch >> >> >>>> seems to not care about version requirements, which are crucial for >> >> >>>> me. >> >> >>>> Maybe that is something to reconsider before merging. >> >> >>>> >> >> >>>> Best, >> >> >>>> Jan >> >> >>>> >> >> >>>> pá 14. 10. 2022 v 2:27 odesílatel Jan Gorecki <[email protected]> >> >> >>>> napsal: >> >> >>>> >> >> >>>>> Dear R devs, >> >> >>>>> >> >> >>>>> I would like to raise a request for a simple helper function. >> >> >>>>> Utility function to extract package dependencies from DESCRIPTION >> >> >>>>> file. >> >> >>>>> >> >> >>>>> I do think that tools package is better place, for such a >> >> >>>>> fundamental >> >> >>>>> functionality, than community packages. >> >> >>>>> >> >> >>>>> tools pkg seems perfect fit (having already great function >> >> >>>>> write_PACKAGES). >> >> >>>>> >> >> >>>>> Functionality I am asking for is already in R svn repository since >> >> >>> 2016, >> >> >>>>> in >> >> >>>>> a branch tools4pkgs. Function is called 'packages.dcf'. >> >> >>>>> Another one 'repos.dcf' would be a good functional complementary to >> >> >>>>> it. >> >> >>>>> >> >> >>>>> Those two simple helper functions really makes it easier for >> >> >>> organizations >> >> >>>>> to glue together usage of their own R packages repos and CRAN repo >> >> >>>>> in a >> >> >>>>> smooth way. That could possibly help to offload CRAN from new >> >> >>> submissions. >> >> >>>>> >> >> >>>>> gh mirror link for easy preview: >> >> >>>>> >> >> >>>>> >> >> >>> https://github.com/wch/r-source/blob/tools4pkgs/src/library/tools/R/packages.R#L419 >> >> >>>>> >> >> >>>>> Regards >> >> >>>>> Jan Gorecki >> >> >>>>> >> >> >>>>> [[alternative HTML version deleted]] >> >> >>>>> >> >> >>>>> ______________________________________________ >> >> >>>>> [email protected] mailing list >> >> >>>>> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> >>>>> >> >> >>>> >> >> >>> >> >> >>> [[alternative HTML version deleted]] >> >> >>> >> >> >>> ______________________________________________ >> >> >>> [email protected] mailing list >> >> >>> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> >>> >> >> >> >> >> > >> >> > [[alternative HTML version deleted]] >> >> > >> >> > ______________________________________________ >> >> > [email protected] mailing list >> >> > https://stat.ethz.ch/mailman/listinfo/r-devel >> >> > >> >> ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
