Re: [R-pkg-devel] Win-builder not finding binaries provided by LinkingTo: cargo?
My DESCRIPTION has the same “SystemRequirements: …” line as salso—from which I probably copied it. I had briefly reached out to ‘cargo’ author (who is also ‘salso’ author), who indicated I might need to supply binaries. Indeed, on closer look, salso appears to include the binaries, rather than asking CRAN to do the build: https://www.r-project.org/nosvn/R.check/r-devel-windows-x86_64-gcc10-UCRT/salso-00check.html Thus, it seems ‘precautionary’ presents an opportunity to get Rust compilation working on Win-builder. Rust is such a fine language that it would be a shame not to smooth this out for the community. OTOH, crucial calculations in ‘precautionary’ parallelize so nicely with mclapply() that it would not be inappropriate to bypass (forkless) Win by setting OS_type to unix. From: Kevin Ushey Date: Tuesday, April 6, 2021 at 6:41 PM To: David Norris Cc: "r-package-devel@r-project.org" Subject: Re: [R-pkg-devel] Win-builder not finding binaries provided by LinkingTo: cargo? I think this is the pertinent error: make: cargo: Command not found That is, make is assuming that the cargo utility is installed and available on the PATH, but that doesn't appear to be true on the CRAN winbuilder machine. The salso package (https://cran.r-project.org/package=salso) explicitly requires cargo to be already available, per SystemRequirements: SystemRequirements: Cargo (>= 1.42.0) for installation from sources: see file INSTALL so you probably need to do something similar. I'd also recommend downloading the 'salso' package to see how they resolve this, or contacting the 'salso' maintainer to see what steps they took to ensure their package could build on CRAN. Best, Kevin On Tue, Apr 6, 2021 at 12:10 PM David Norris mailto:da...@precisionmethods.guru>> wrote: For version 0.2-2 of my ‘precautionary’ package, I have added fast numerical routines implemented in Rust. In my DESCRIPTION file, I am “LinkingTo: cargo (>= 0.1.28)”, but fail incoming check on Win-builder as follows: https://win-builder.r-project.org/incoming_pretest/precautionary_0.2-2_20210406_142130/Windows/00install.out * installing *source* package 'precautionary' ... ** using staged installation ** libs *** arch - i386 rm -Rf precautionary.dll ./rust/target/i686-pc-windows-gnu/release/libprecautionary.a entrypoint.o d:/Compiler/rtools40/mingw32/bin/gcc -I"D:/RCompile/recent/R/include" -DNDEBUG -I'd:/RCompile/CRANpkg/lib/4.1/cargo/include' -I"d:/Compiler/gcc-4.9.3/local330/include" -pedantic -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c entrypoint.c -o entrypoint.o cargo build --target=i686-pc-windows-gnu --lib --release --manifest-path=./rust/Cargo.toml make: cargo: Command not found make: *** [Makevars.win:11: rust/target/-pc-windows-gnu/release/libprecautionary.a] Error 127 ERROR: compilation failed for package 'precautionary' * removing 'd:/RCompile/CRANincoming/R-devel/lib/precautionary' As I understand the intention of package cargo https://CRAN.R-project.org/package=cargo, it aims to solve precisely this problem. Other platforms check my package just fine. Is the solution to be found in configuration of Win-builder, in my Makevars.win, or elsewhere? I have taken my Makevars.win straight from https://github.com/extendr/helloextendr/blob/main/src/Makevars.win as follows: TARGET = $(subst 64,x86_64,$(subst 32,i686,$(WIN)))-pc-windows-gnu LIBDIR = ./rust/target/$(TARGET)/release STATLIB = $(LIBDIR)/libprecautionary.a PKG_LIBS = -L$(LIBDIR) -lprecautionary -lws2_32 -ladvapi32 -luserenv all: C_clean $(SHLIB): $(STATLIB) $(STATLIB): cargo build --target=$(TARGET) --lib --release --manifest-path=./rust/Cargo.toml C_clean: rm -Rf $(SHLIB) $(STATLIB) $(OBJECTS) clean: rm -Rf $(SHLIB) $(STATLIB) $(OBJECTS) rust/target [[alternative HTML version deleted]] __ R-package-devel@r-project.org<mailto:R-package-devel@r-project.org> mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Best way to build CRAN packages that include Rust source?
I am appreciating 2 distinct approaches currently used to compile Rust code in R packages: 1. LinkingTo: cargo, as done in https://CRAN.R-project.org/package=salso 2. A more ‘traditional’-looking approach built on Autoconf & Automake https://CRAN.R-project.org/package=gifski Is there any reason why one or the other would be preferred for a CRAN submission? Would one or the other design better facilitate Rust’s eventual elevation to ‘first-class’ status (like C++, Fortran) for compiling code in R packages? In favor of (1), it seems to me that the “Reverse linking to:” entry at https://CRAN.R-project.org/package=cargo nicely advertises packages that use Rust, which could be useful for example code, etc. In favor of (2), it seems completely idiomatic—and so potentially more conducive to achieving first-class status for Rust. [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Best way to build CRAN packages that include Rust source?
To follow up, I’m quite settled on (2) now, as I’ve been able to adapt the Makevars files of package ‘gifski’ to achieve a successful R-CMD-check across multiple GitHub virtual platforms including windows. For anyone interested, the Makevars & Makevars.win files are in https://github.com/dcnorris/precautionary/tree/main/src The GitHub workflow is at https://github.com/dcnorris/precautionary/blob/main/.github/workflows/R-CMD-check.yaml Kind regards, David From: R-package-devel on behalf of David Norris Date: Monday, May 3, 2021 at 2:44 PM To: "r-package-devel@r-project.org" Subject: [R-pkg-devel] Best way to build CRAN packages that include Rust source? I am appreciating 2 distinct approaches currently used to compile Rust code in R packages: 1. LinkingTo: cargo, as done in https://CRAN.R-project.org/package=salso 2. A more ‘traditional’-looking approach built on Autoconf & Automake https://CRAN.R-project.org/package=gifski Is there any reason why one or the other would be preferred for a CRAN submission? Would one or the other design better facilitate Rust’s eventual elevation to ‘first-class’ status (like C++, Fortran) for compiling code in R packages? In favor of (1), it seems to me that the “Reverse linking to:” entry at https://CRAN.R-project.org/package=cargo nicely advertises packages that use Rust, which could be useful for example code, etc. In favor of (2), it seems completely idiomatic—and so potentially more conducive to achieving first-class status for Rust. [[alternative HTML version deleted]] __ R-package-devel@r-project.org<mailto:R-package-devel@r-project.org> mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Using parallel:::sendMaster &c. despite warnings
Because parallelized progress reporting in the futureverse.org incurs latencies too great for my application (https://github.com/HenrikBengtsson/progressr/issues/118), I have found it necessary to implement my own progress reporting using some of the non-exported functionality from `parallel`. (I do appreciate that Windows lacks the fork() system call, and will not support this. But am willing to make this an OS_type: unix-only package.) Of course, I get a WARNING for this: ── R CMD check results precautionary 0.2.6 Duration: 6m 41.8s ❯ checking dependencies in R code ... WARNING Unexported objects imported by ':::' calls: ‘parallel:::readChild’ ‘parallel:::selectChildren’ ‘parallel:::sendMaster’ See the note in ?`:::` about the use of this operator. Including base/recommended package(s): ‘parallel’ Is this warning an absolute deal-killer on CRAN? Is there a 'correct' way to do `:::` that avoids the warning altogether? Kind regards, David Norris __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings
I can confirm that getFromNamespace() does indeed circumvent the WARNING. I might otherwise prefer ':::', however, for making its 'bad intentions' so clear. I was very much aware of the cautionary language under `?readChild`, but wondered whether *in practice* more liberal community policies might be in force pending the maturation of the futureverse. Although the sparse history of the relevant file hardly supports "might-break-at-any-time" alarmism, I do note that the last commit (in May 2020) introduced an argument sendMaster(raw.asis=TRUE) that I am actually using. https://github.com/wch/r-source/commits/trunk/src/library/parallel/R/unix/mcfork.R (Also agree with Yohann that ideally one might support Windows with a warning such as data.table issues regarding my Mac's non-support for OpenMP.) Best, David From: Sebastian Meyer Organization: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Date: Friday, July 23, 2021 at 8:05 AM To: David Norris , "r-package-devel@r-project.org" Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings Am 23.07.21 um 13:19 schrieb David Norris: Because parallelized progress reporting in the futureverse.org incurs latencies too great for my application (https://github.com/HenrikBengtsson/progressr/issues/118), I have found it necessary to implement my own progress reporting using some of the non-exported functionality from `parallel`. (I do appreciate that Windows lacks the fork() system call, and will not support this. But am willing to make this an OS_type: unix-only package.) Of course, I get a WARNING for this: ── R CMD check results precautionary 0.2.6 Duration: 6m 41.8s ❯ checking dependencies in R code ... WARNING Unexported objects imported by ':::' calls: ‘parallel:::readChild’ ‘parallel:::selectChildren’ ‘parallel:::sendMaster’ See the note in ?`:::` about the use of this operator. Including base/recommended package(s): ‘parallel’ Is this warning an absolute deal-killer on CRAN? Is there a 'correct' way to do `:::` that avoids the warning altogether? The 'parallel' functions your package intends to access seem to be intentionally unexported. Their help page says: "They are not available on Windows, and not exported from the namespace", and "This is a very low-level interface for expert use only: it not regarded as part of the R API and subject to change without notice." Correspondingly, the CRAN Repository Policy says Also, ::: should not be used to access undocumented/internal objects in base packages (nor should other means of access be employed). Such usages can cause packages to break at any time, even in patched versions of R. which kind of answers both of your questions. The policy thus implicitly advises against using getFromNamespace(). Best regards, Sebastian Meyer Kind regards, David Norris __ mailto:R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings
My understanding about this issue with progressr is that it is nontrivial, and will take author Henrik Bengtsson some time & thought to resolve. OTOH if patches to 'parallel' were welcome, I could embed my solution in a modified version of parallel::mclapply by introducing 1 or 2 new parameters (e.g., mc.prog=NULL) to support progress reporting consistently with the existing interface. From: Duncan Murdoch Date: Friday, July 23, 2021 at 11:26 AM To: Hugh Parsonage , David Norris Cc: "r-package-devel@r-project.org" Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings On 23/07/2021 10:59 a.m., Hugh Parsonage wrote: Happily, package parallel is open-source. If you would like to make use of the unexported functions, you could copy the source code with acknowledgement into your package. There's a good chance that wouldn't work, because parallel is a base package. Base packages can work with R internals and user-written packages can't. Probably David's best course of action is to submit patches where necessary so that he doesn't need ::: access. The original motivation appeared to be avoiding inefficiency in a contributed package; if there's a patch that can fix that, it could be available in a few days. If it needs changes to the parallel package (e.g. exposing functions that are currently internal), that will take much longer, and needs a strong argument why the current API isn't sufficient. Alternatively, he can go ahead and use :::, but just not expect his package to be on CRAN. There are other ways to distribute packages. Duncan Murdoch Naturally this might be a lot of work and risks different behaviour between your package and parallel, but neither might be that terrible for you. And certainly would be better than trying to pretend that the functions you want are exported from parallel. On Fri, 23 Jul 2021 at 23:09, David Norris <mailto:da...@precisionmethods.guru> wrote: I can confirm that getFromNamespace() does indeed circumvent the WARNING. I might otherwise prefer ':::', however, for making its 'bad intentions' so clear. I was very much aware of the cautionary language under `?readChild`, but wondered whether *in practice* more liberal community policies might be in force pending the maturation of the futureverse. Although the sparse history of the relevant file hardly supports "might-break-at-any-time" alarmism, I do note that the last commit (in May 2020) introduced an argument sendMaster(raw.asis=TRUE) that I am actually using. https://github.com/wch/r-source/commits/trunk/src/library/parallel/R/unix/mcfork.R (Also agree with Yohann that ideally one might support Windows with a warning such as data.table issues regarding my Mac's non-support for OpenMP.) Best, David From: Sebastian Meyer <mailto:seb.me...@fau.de> Organization: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Date: Friday, July 23, 2021 at 8:05 AM To: David Norris <mailto:da...@precisionmethods.guru>, "mailto:r-package-devel@r-project.org"; <mailto:r-package-devel@r-project.org> Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings Am 23.07.21 um 13:19 schrieb David Norris: Because parallelized progress reporting in the futureverse.org incurs latencies too great for my application (https://github.com/HenrikBengtsson/progressr/issues/118), I have found it necessary to implement my own progress reporting using some of the non-exported functionality from `parallel`. (I do appreciate that Windows lacks the fork() system call, and will not support this. But am willing to make this an OS_type: unix-only package.) Of course, I get a WARNING for this: ── R CMD check results precautionary 0.2.6 Duration: 6m 41.8s ❯ checking dependencies in R code ... WARNING Unexported objects imported by ':::' calls: ‘parallel:::readChild’ ‘parallel:::selectChildren’ ‘parallel:::sendMaster’ See the note in ?`:::` about the use of this operator. Including base/recommended package(s): ‘parallel’ Is this warning an absolute deal-killer on CRAN? Is there a 'correct' way to do `:::` that avoids the warning altogether? The 'parallel' functions your package intends to access seem to be intentionally unexported. Their help page says: "They are not available on Windows, and not exported from the namespace", and "This is a very low-level interface for expert use only: it not regarded as part of the R API and subject to change without notice." Correspondingly, the CRAN Repository Policy says Also, ::: should not be used to access undocumented/internal objects in base packages (nor should other means of access be employed). Such usages can cause packages to break at any time, even in p
Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings
Thanks for delineating r-devel vs this list, Ben, and for invitation to start a fresh convo over there. I'll do that once I can exhibit a basic proposal functioning within package 'precautionary'. Best, David From: R-package-devel on behalf of Ben Bolker Date: Friday, July 23, 2021 at 1:29 PM To: "r-package-devel@r-project.org" Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings Patches to parallel could certainly be *considered*, but people will probably want to argue about the details quite a bit. I would suggest presenting/discussing your ideas for modification on r-devel before posting them as a 'wishlist' item to the R bug tracker. (If an R-core member participates in the discussion on r-devel and is convinced of the utility and harmlessness of your modifications the last step might be unnecessary). cheers Ben Bolker On 7/23/21 1:17 PM, David Norris wrote: My understanding about this issue with progressr is that it is nontrivial, and will take author Henrik Bengtsson some time & thought to resolve. OTOH if patches to 'parallel' were welcome, I could embed my solution in a modified version of parallel::mclapply by introducing 1 or 2 new parameters (e.g., mc.prog=NULL) to support progress reporting consistently with the existing interface. From: Duncan Murdoch <mailto:murdoch.dun...@gmail.com> Date: Friday, July 23, 2021 at 11:26 AM To: Hugh Parsonage <mailto:hugh.parson...@gmail.com>, David Norris <mailto:da...@precisionmethods.guru> Cc: "mailto:r-package-devel@r-project.org"; <mailto:r-package-devel@r-project.org> Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings On 23/07/2021 10:59 a.m., Hugh Parsonage wrote: Happily, package parallel is open-source. If you would like to make use of the unexported functions, you could copy the source code with acknowledgement into your package. There's a good chance that wouldn't work, because parallel is a base package. Base packages can work with R internals and user-written packages can't. Probably David's best course of action is to submit patches where necessary so that he doesn't need ::: access. The original motivation appeared to be avoiding inefficiency in a contributed package; if there's a patch that can fix that, it could be available in a few days. If it needs changes to the parallel package (e.g. exposing functions that are currently internal), that will take much longer, and needs a strong argument why the current API isn't sufficient. Alternatively, he can go ahead and use :::, but just not expect his package to be on CRAN. There are other ways to distribute packages. Duncan Murdoch Naturally this might be a lot of work and risks different behaviour between your package and parallel, but neither might be that terrible for you. And certainly would be better than trying to pretend that the functions you want are exported from parallel. On Fri, 23 Jul 2021 at 23:09, David Norris <mailto:da...@precisionmethods.guru> wrote: I can confirm that getFromNamespace() does indeed circumvent the WARNING. I might otherwise prefer ':::', however, for making its 'bad intentions' so clear. I was very much aware of the cautionary language under `?readChild`, but wondered whether *in practice* more liberal community policies might be in force pending the maturation of the futureverse. Although the sparse history of the relevant file hardly supports "might-break-at-any-time" alarmism, I do note that the last commit (in May 2020) introduced an argument sendMaster(raw.asis=TRUE) that I am actually using. https://github.com/wch/r-source/commits/trunk/src/library/parallel/R/unix/mcfork.R (Also agree with Yohann that ideally one might support Windows with a warning such as data.table issues regarding my Mac's non-support for OpenMP.) Best, David From: Sebastian Meyer <mailto:seb.me...@fau.de> Organization: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Date: Friday, July 23, 2021 at 8:05 AM To: David Norris <mailto:da...@precisionmethods.guru>, "mailto:r-package-devel@r-project.org"; <mailto:r-package-devel@r-project.org> Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings Am 23.07.21 um 13:19 schrieb David Norris: Because parallelized progress reporting in the futureverse.org incurs latencies too great for my application (https://github.com/HenrikBengtsson/progressr/issues/118), I have found it necessary to implement my own progress reporting using some of the non-exported functionality from `parallel`. (I do appreciate that Windows lacks the fork() system call, and will not support this. But am willing to make this an OS_type: unix-only package.) Of course, I get a WARNING for this: ── R CMD check results
[R-pkg-devel] UX for "WARNING unable to translate to native encoding"?
Whereas Unicode U+00d7 (times), U+00b1 (plus-minus) and U+03bc (mu) have equivalents in Latin-1 encoding, and I have used these without difficulty in strings, neither U+2206 (INCREMENT) nor U+0394 (Greek Delta) does: https://en.wikipedia.org/wiki/ISO/IEC_8859-1 Consequently, my attempt to use a parameter named '\u2206t' as in the following > attr(path_list,'performance') <- self$report(J = length(path_list) >, t1 = round(t1) >, t2 = round(t2) >, "\u2206t" = round(t2 - > t1) > ) generates the following WARNING on Windows R-CMD-check: > checking R files for syntax errors ... WARNING Warning in file 'R/cpe.R': unable to translate 't' to native encoding >From this article by Tomas Kalibera https://developer.r-project.org/Blog/public/2020/05/02/utf-8-support-on-windows/, I gather that quite often problems such as these won't actually bubble up to the user experience. Is this one such case? Or would this WARNING be fatal to a CRAN submission? __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] UX for "WARNING unable to translate to native encoding"?
Thanks to you both, Ivan and Tomas. Capital D it is, then. -David From: Tomas Kalibera Date: Monday, August 16, 2021 at 8:48 AM To: Ivan Krylov , David Norris Cc: "r-package-devel@r-project.org" Subject: Re: [R-pkg-devel] UX for "WARNING unable to translate to native encoding"? On 8/16/21 12:42 PM, Ivan Krylov wrote: On Mon, 16 Aug 2021 09:05:54 + David Norris <mailto:da...@precisionmethods.guru> wrote: Unicode U+00d7 (times), U+00b1 (plus-minus) and U+03bc (mu) have equivalents in Latin-1 encoding, and I have used these without difficulty in strings, neither U+2206 (INCREMENT) nor U+0394 (Greek Delta) does But not in some other locale encodings on Windows (e.g. CP-1251), nor in some single-byte locale encodings on *nix-like systems (e.g. ru_RU.KOI8-R), which are admittedly used much rarer nowadays than on Windows. Unless I'm mistaken, the "\u2206t" in your example needs to become a symbol, and symbols are always translated into the locale encoding [1] [2]. I would expect this warning to be a problem for CRAN, but I'm just another package developer, so I could be wrong. Yes, this is a problem. Only ASCII characters should be used in symbol names (in R packages), as they can be represented in every (supported) locale. Some characters would be best-fitted by Windows (replaced by other similar characters) during translation to native encoding, if they are not representable directly, but that can produce surprising results and should not be relied on, definitely not in packages. Best Tomas __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Win-builder not finding binaries provided by LinkingTo: cargo?
For version 0.2-2 of my ‘precautionary’ package, I have added fast numerical routines implemented in Rust. In my DESCRIPTION file, I am “LinkingTo: cargo (>= 0.1.28)”, but fail incoming check on Win-builder as follows: https://win-builder.r-project.org/incoming_pretest/precautionary_0.2-2_20210406_142130/Windows/00install.out * installing *source* package 'precautionary' ... ** using staged installation ** libs *** arch - i386 rm -Rf precautionary.dll ./rust/target/i686-pc-windows-gnu/release/libprecautionary.a entrypoint.o d:/Compiler/rtools40/mingw32/bin/gcc -I"D:/RCompile/recent/R/include" -DNDEBUG -I'd:/RCompile/CRANpkg/lib/4.1/cargo/include' -I"d:/Compiler/gcc-4.9.3/local330/include" -pedantic -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c entrypoint.c -o entrypoint.o cargo build --target=i686-pc-windows-gnu --lib --release --manifest-path=./rust/Cargo.toml make: cargo: Command not found make: *** [Makevars.win:11: rust/target/-pc-windows-gnu/release/libprecautionary.a] Error 127 ERROR: compilation failed for package 'precautionary' * removing 'd:/RCompile/CRANincoming/R-devel/lib/precautionary' As I understand the intention of package cargo https://CRAN.R-project.org/package=cargo, it aims to solve precisely this problem. Other platforms check my package just fine. Is the solution to be found in configuration of Win-builder, in my Makevars.win, or elsewhere? I have taken my Makevars.win straight from https://github.com/extendr/helloextendr/blob/main/src/Makevars.win as follows: TARGET = $(subst 64,x86_64,$(subst 32,i686,$(WIN)))-pc-windows-gnu LIBDIR = ./rust/target/$(TARGET)/release STATLIB = $(LIBDIR)/libprecautionary.a PKG_LIBS = -L$(LIBDIR) -lprecautionary -lws2_32 -ladvapi32 -luserenv all: C_clean $(SHLIB): $(STATLIB) $(STATLIB): cargo build --target=$(TARGET) --lib --release --manifest-path=./rust/Cargo.toml C_clean: rm -Rf $(SHLIB) $(STATLIB) $(OBJECTS) clean: rm -Rf $(SHLIB) $(STATLIB) $(OBJECTS) rust/target [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
