Re: [Rd] Packages manually setting NeedsCompilation
On Wed, May 14, 2025 at 11:05 PM Simon Urbanek wrote: > > Jeroen, > > thanks for raising the issues. Comments inline. > > > > On May 15, 2025, at 1:47 AM, Jeroen Ooms wrote: > > > > R-universe builds and checks all CRAN packages on arm64 on Mac, Linux > > and soon Windows. It is important that we can identify from a binary > > package for which architecture it was built. R inserts this > > information into the second part of the "Built:" field in the > > DESCRIPTION. For example, packages with compiled code contain: > > > >Built: R 4.5.0; aarch64-unknown-linux-gnu; 2025-05-05 14:06:56 UTC; unix > > > > And packages without compiled code contain: > > > >Built: R 4.5.0; ; 2025-05-08 06:46:33 UTC; unix > > > > However this is not working for packages that compile something, but > > do not have a 'src' dir. > > > Can you give an example, please? I wonder if there is a real use-case or just > bad package design. The case that bit me yesterday was a Bioconductor package Rigraphlib, see https://bioconductor.org/packages/release/bioc/src/contrib/Rigraphlib_1.0.0.tar.gz This package builds a static library entirely from a configure script. Because there is no 'src' dir, we assumed the package to be all-arch, and get linking errors when trying to use it on another arch then it was built for. Arguably this is indeed bad package design, but that is hard to protect against. > I wouldn't think that should happen as configure is supposed to only guide > the compilation in src - if there is no src no binaries are expected as the > package did not provide any native sources hence there should be no binary > content. This looks like something that could be added to R CMD check? In that case maybe R should warn if it runs a package configure script but there is no "src"? And check if NeedsCompilation: yes is set? One way or another, what matters is that when R runs configure or make during install, the value in the "Built" field in the binary package should contain the platform value. This is also important when distros ship R packages in deb or rpm format, to prevent packages that have compiled code from getting "Architecture: all" and be installed on the wrong platforms. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Packages manually setting NeedsCompilation
On 15 May 2025 at 11:41, Jeroen Ooms wrote: | The case that bit me yesterday was a Bioconductor package Rigraphlib, | see https://bioconductor.org/packages/release/bioc/src/contrib/Rigraphlib_1.0.0.tar.gz | | This package builds a static library entirely from a configure script. | Because there is no 'src' dir, we assumed the package to be all-arch, | and get linking errors when trying to use it on another arch then it | was built for. Arguably this is indeed bad package design, but that is | hard to protect against. | | > I wouldn't think that should happen as configure is supposed to only guide the compilation in src - if there is no src no binaries are expected as the package did not provide any native sources hence there should be no binary content. This looks like something that could be added to R CMD check? | | In that case maybe R should warn if it runs a package configure script | but there is no "src"? And check if NeedsCompilation: yes is set? Conceivably we may have valid use cases to run 'detection' in `configure` that is not related to src/ and compilation. | One way or another, what matters is that when R runs configure or make | during install, the value in the "Built" field in the binary package | should contain the platform value. This is also important when distros | ship R packages in deb or rpm format, to prevent packages that have | compiled code from getting "Architecture: all" and be installed on the | wrong platforms. That view is fairly narrow as it affects only R CMD INSTALL --build use (which may of course be the most common by far). But eg r2u wraps .deb meta-information around the inner tar.gz it does so by relying on src/ being present to detect binary per-platform status (Architecture: any" in Debian parlance). Can we get this BioConductor package to add a src/ field? Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Packages manually setting NeedsCompilation
On Thu, 15 May 2025 at 11:42, Jeroen Ooms wrote: > > On Wed, May 14, 2025 at 11:05 PM Simon Urbanek > wrote: > > > > Can you give an example, please? I wonder if there is a real use-case or > > just bad package design. > > The case that bit me yesterday was a Bioconductor package Rigraphlib, > see > https://bioconductor.org/packages/release/bioc/src/contrib/Rigraphlib_1.0.0.tar.gz > > This package builds a static library entirely from a configure script. > Because there is no 'src' dir, we assumed the package to be all-arch, > and get linking errors when trying to use it on another arch then it > was built for. Arguably this is indeed bad package design, but that is > hard to protect against. The opposite case is also true: there is a src directory but no compilation happens. This is a problem for deb/rpm shipment for two reasons. In rpm packaging at least, archful packages are inspected to extract debuginfo; since there is no compiled output there, this fails and the whole build fails. If the build succeeds (e.g. debuginfo is disabled), then we are building the same thing for all architectures, when there should be just one, noarch, for all. I can list at least 3 examples of this: - BANOVA: requires compilation, there is a src directory, nothing happens there, no source files, just a Makefile sitting there. - mixl: same, and the Makefile is just used to print if OpenMP is supported (?). - mathjaxr: same, and the Makefile is used to install a javascript library (!). > > I wouldn't think that should happen as configure is supposed to only guide > > the compilation in src - if there is no src no binaries are expected as the > > package did not provide any native sources hence there should be no binary > > content. This looks like something that could be added to R CMD check? > > In that case maybe R should warn if it runs a package configure script > but there is no "src"? And check if NeedsCompilation: yes is set? > > One way or another, what matters is that when R runs configure or make > during install, the value in the "Built" field in the binary package > should contain the platform value. This is also important when distros > ship R packages in deb or rpm format, to prevent packages that have > compiled code from getting "Architecture: all" and be installed on the > wrong platforms. IMHO, the flag NeedsCompilation in the DESCRIPTION file should be the one and only source of truth here, regardless of the presence or not of a src directory. Then I see two paths here: - Either the developer is responsible for setting this flag, and the R CMD check should fail if it's yes but no compilation happens or the other way around. - Or R CMD build is responsible for setting this flag after checking if compilation happens or not. -- Iñaki Úcar __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Packages manually setting NeedsCompilation
> Simon Urbanek writes: Re part II (CRAN packages which have NeedsCompilation: yes but don't really need it): I of course have code for this, last touched in 2016. Running the code (which currently only looks at having a 'src' subdir, based on one possible interpreation of it is normally set by ‘R CMD build’ or the repository assuming compilation is required if and only if the package has a ‘src’ directory. in WRE) finds the same packages as on Jeroen's list (minus bcgam and NetFACS which got archived recently). One could integrate a variant of the code into R CMD check if we think there is a "real need" (which I admit I have not seen thus far or else I would have added check code). For now I'll try the effect of writing to the maintainers of the 124 current CRAN packages which have NeedsCompilation: yes but no 'src' and asking them to fix "as necessary", without setting a formal deadline. Best -k > Jeroen, > thanks for raising the issues. Comments inline. >> On May 15, 2025, at 1:47 AM, Jeroen Ooms wrote: >> >> R-universe builds and checks all CRAN packages on arm64 on Mac, Linux >> and soon Windows. It is important that we can identify from a binary >> package for which architecture it was built. R inserts this >> information into the second part of the "Built:" field in the >> DESCRIPTION. For example, packages with compiled code contain: >> >> Built: R 4.5.0; aarch64-unknown-linux-gnu; 2025-05-05 14:06:56 UTC; unix >> >> And packages without compiled code contain: >> >> Built: R 4.5.0; ; 2025-05-08 06:46:33 UTC; unix >> >> However this is not working for packages that compile something, but >> do not have a 'src' dir. > Can you give an example, please? I wonder if there is a real use-case or just > bad package design. I wouldn't think that should happen as configure is > supposed to only guide the compilation in src - if there is no src no > binaries are expected as the package did not provide any native sources hence > there should be no binary content. This looks like something that could be > added to R CMD check? >> I think these packages are supposed to manually set "NeedsCompilation: yes" >> in their DESCRIPTION file. Two suggestions: >> >> 1. Currently R only records the platform in the mentioned "Built" if >> the source package contains a 'src' dir. Could this be improved such >> that it also does so for these packages with "NeedsCompilation: yes"? >> Here is a two line patch: >> https://github.com/r-devel/r-svn/pull/199/files >> >> 2. There are a handful of CRAN packages that have "NeedsCompilation: >> yes" but they neither have a 'src' dir nor a 'configure' script. Afaik >> this is a contradiction, as there is no place any compilation can be >> invoked without either of these files present? I think in most these >> cases "NeedsCompilation: yes" is a leftover from older versions. If >> so, would it be possible for CMD check to warn against redundant >> NeedsCompilation:yes? Here is a script to list packages that currently >> use this: https://gist.github.com/jeroen/e8b225e9b1004a16ef90f50700352aa5 >> > I think both cases look like candidates to R CMD check, so, yes, agreed. > Without valid use-cases I'm not convinced yet that an override is a good idea > since it may get out of sync just as you noted, but I may be just missing a > valid use-case. That said, if the override is paired with the check, i.e., if > both your proposals are implemented (and a warning if configure generates > binaries without declaring them) that would cover all the bases. > Cheers, > Simon > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
