Re: [Rd] OpenBLAS in everyday R?

[Avraham Adler]

On Wed, Jan 10, 2018 at 12:04 AM, Keith O'Hara  wrote:
>
> Check if libopenblas is linked against libomp or libgomp.
>
> I’d be curious to see any errors that arise when an OpenMP version of 
> OpenBLAS is linked with R.
>
> Keith
>

The one time I tried compiling OpenBLAS for Windows 64 with USE OMP =
1, I got an error. I don't recall if it was in the compilation of R or
in use. Regardless, I compile OpenBLAS without setting that flag and
it still plays nicely with all R packages, including those that use
OpenMP.

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

[Avraham Adler]

On Thu, Jan 11, 2018 at 10:38 AM, Keith O'Hara  wrote:

[snip]
>
> Perhaps another point for Juan’s list: whether OpenBLAS is the right choice 
> to pair with. The library itself hasn’t produced optimized kernels for any of 
> the Intel *Lake chips yet; might be worth considering its near- and long-term 
> future (vs something else).
>

Regarding this point, please see this thread on OpenBLAS-users [1], in
specific the third post by Jeff Hammond who says he is an employee of
Intel, where he says:

 "Skylake Xeon processors with AVX-512 are definitely going to require
code changes to perform optimally. However, the Core i[357] processors
up to and including Kaby Lake do not support AVX-512 and thus can
suffice with the existing AVX2 implementation that targets Haswell.
See https://ark.intel.com/#@Processors and look for "Instruction Set
Extensions" for full details on on specific SKUs."

I presume this holds for CoffeeLake as well, as it's pretty much a
hexacore SkyLake.

Thank you,

Avi

[1] https://groups.google.com/d/msg/openblas-users/XU6-9h-geVE/mwz2ewKrCQAJ

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Compilation gets stuck on Windows 64

[Avraham Adler]

On Thu, Feb 8, 2018 at 9:44 PM, Indrajit Sen Gupta  wrote:
> Hi All,
>
> I am trying to compile R from source on a 64 bit Windows.

[snip]

> I had compiled earlier with MinGW and had created the file:
> *libopenblas_haswell-r0.2.20.a. *

Hello, Indrajit.

I don't see your MkRules.local attached. In any event, perhaps try
following the directions here [1]. I've been building R with OpenBLAS
on Windows 64 for years and it almost always works. In the past year
or two, rarely, it will stop with an error. But if you restart the
make process (by just typing "make" again) it finishes with no issues
and passes make check-devel. I have not tried this with R-dev, though.
R 3.4.3 Patched (2018-01-03 r74042) is the most recent I have built
successfully.

Good luck,

Avraham

[1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Compilation gets stuck on Windows 64

[Avraham Adler]

On Fri, Feb 9, 2018 at 2:16 AM, Indrajit Sen Gupta  wrote:
> Hi Avraham,
>
> A quick question - I realized I did not have Perl installed. So I installed
> ActiveState Perl right now. Also I see I need texinfo and texi2any. I was
> able to installed texinfo from here:
> http://gnuwin32.sourceforge.net/packages/texinfo.htm. But not sure where to
> get texi2any. Can you guide me in this step?

It is in the ZIP file "texinfo5.zip" here [1]. Unzip that entire file
into a directory and use that as your texinfo directory. That works
for me.

Avi

[1] http://www.stats.ox.ac.uk/pub/Rtools/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Compilation gets stuck on Windows 64

[Avraham Adler]

On Fri, Feb 9, 2018 at 9:29 AM, Kenny Bell  wrote:
> I suggest that you work off the build process in the  rwinlib repository so
> you are starting from something that you know works and already incorporates
> the set of dependencies you need.

Hello, Kenny.

For what it's worth I've been successfully building R+OpenBLAS on
Windows64 since 2013, which I believe predates rwinlib on github, but
I may be mistaken. Thus, I don't think I'm incorrect in saying that
the instructions I provide are also "something that you know works" :)
I did make the explicit assumption that people will successfully
follow the instructions at R Installation & Administration. Perhaps
that is too much to ask.

Thanks,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Compilation gets stuck on Windows 64

[Avraham Adler]

Set the location of atlas path in mkrules.local to what you need, then edit
arc/extra/blas/makevars.win to read lopenblas_haswell-r0.2.20 instead of
the two libraries there in the atlas section. I think I described this in
my more recent instructions.

Avi

On Tue, Feb 13, 2018 at 8:18 AM Indrajit Sen Gupta 
wrote:

> In the file MkRules.local.in, I see the line: USE_ATLAS = NO which I
> believe needs to be changed to YES. But how do I specify the BLAS file 
> *libopenblas_haswell-r0.2.20.a
> *and its location?
>
> Regards,
> Indrajit
>
> On Tue, Feb 13, 2018 at 6:41 PM, Indrajit Sen Gupta 
> wrote:
>
>> I was able to compile the R from the github by running build-r-devel.bat!
>>
>> Now need to see how to compile it with BLAS.
>>
>> Regard,
>> Indrajit
>>
>> On Tue, Feb 13, 2018 at 5:45 PM, Jeroen Ooms 
>> wrote:
>>
>>> On Tue, Feb 13, 2018 at 12:22 PM, Indrajit Sen Gupta
>>>  wrote:
>>> > Hi Avraham,
>>> >
>>> > I tried with the patched version. The same error message.
>>> >
>>> > gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
>>> dynload.o
>>> > editor.o embeddedR.o extra.o malloc.o opt.o pager.o preferences.o
>>> psignal.o
>>> > rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o system.o
>>> dos_wglob.o
>>> > dllversion.o ../main/libmain.a ../appl/libappl.a ../nmath/libnmath.a
>>> > getline/gl.a ../extra/xdr/libxdr.a ../extra/intl/libintl.a
>>> > ../extra/trio/libtrio.a ../extra/tzone/libtz.a ../extra/tre/libtre.a
>>> > -fopenmp -L. -lgfortran -lquadmath -lRblas -L../../lib -lRgraphapp
>>> -lRiconv
>>> > -lcomctl32 -lversion -L"D:/R64/extsoft"/lib/x64 -lpcre -lz -lbz2 -llzma
>>> > -L"D:/home/ICU"/lib/x64 -lsicuin -lsicuuc -lsicudt -lstdc++
>>> >
>>> D:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe:
>>> > cannot find -lRgraphapp
>>> > collect2.exe: error: ld returned 1 exit status
>>> > make[4]: *** [R.dll] Error 1
>>> > make[3]: *** [../../bin/x64/R.dll] Error 2
>>> > make[2]: *** [rbuild] Error 2
>>> > make[1]: *** [all] Error 2
>>> > make: *** [distribution] Error 2
>>> >
>>> >
>>> > Would it be possible for you to share your MkRules.local and
>>> Makefile.win
>>> > files?
>>>
>>>
>>> Hi Indrajit
>>>
>>> As somebody above already mentioned, the full build script as well as
>>> MkRules.local that we use for the CRAN releases of R for windows are
>>> available from https://github.com/rwinlib/base
>>>
>>> As is explained in the repository readme, if you have the required
>>> dependencies (rtools, miktex innosetup, strawberry perl) all you need
>>> to do is run the build-r-devel.bat script from the root of the
>>> repository.
>>>
>>> Once you got this to work, you can adapt it to your needs.
>>>
>>
>>
> --
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] predict.glm returns different results for the same model

[Avraham Adler]

Can’t copy from my computer as gmail is blocked at work but if it helps,
the “predvars” attribute if the terms object is not the same.

Avi

On Fri, Apr 27, 2018 at 9:25 AM Hadley Wickham  wrote:

> Hi all,
>
> Very surprising (to me!) and mystifying result from predict.glm(): the
> predictions vary depending on whether or not I use ns() or
> splines::ns(). Reprex follows:
>
> library(splines)
>
> set.seed(12345)
> dat <- data.frame(claim = rbinom(1000, 1, 0.5))
> mns <- c(3.4, 3.6)
> sds <- c(0.24, 0.35)
> dat$wind <- exp(rnorm(nrow(dat), mean = mns[dat$claim + 1], sd =
> sds[dat$claim + 1]))
> dat <- dat[order(dat$wind), ]
>
> m1 <- glm(claim ~ ns(wind, df = 5), data = dat, family = binomial)
> m2 <- glm(claim ~ splines::ns(wind, df = 5), data = dat, family = binomial)
>
> # The model coefficients are the same
> unname(coef(m1))
> #> [1]  0.5194712 -0.8687737 -0.6803954  4.0838947  2.3908674  4.1564128
> unname(coef(m2))
> #> [1]  0.5194712 -0.8687737 -0.6803954  4.0838947  2.3908674  4.1564128
>
> # But the predictions are not!
> newdf <- data.frame(wind = seq(min(dat$wind), max(dat$wind), length = 5))
> unname(predict(m1, newdata = newdf))
> #> [1] 0.51947119 0.03208719 2.82548847 3.90883496 4.06743266
> unname(predict(m2, newdata = newdf))
> #> [1]  0.5194712 -0.5666554 -0.1731268  2.8134844  3.9295814
>
> Is this a bug?
>
> (Motivating issue from this ggplot2 bug report:
> https://github.com/tidyverse/ggplot2/issues/2426)
>
> Thanks!
>
> Hadley
>
>
>
> --
> http://hadley.nz
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Issue when building R-parched_2018-12-26

[Avraham Adler]

Hello.

Building R-patched from source on Win64 using current Rtools 3.5.0.4, I’m
getting the following notes indicating an extra left brace somewhere; or is
the problem on my end?

My installed Perl is Strawberry 5.28.0.1-64 bit.

Thank you,

Avi

(Sent from an iPhone, so my apologies if HTML also comes through)

-- Making HTML documentation --

creating doc/manual/version.texi

creating doc/manual/R-FAQ.html

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

creating doc/manual/R-admin.html

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

creating doc/manual/R-data.html

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

creating doc/manual/R-exts.html

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

creating doc/manual/R-intro.html

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

creating doc/manual/R-ints.html

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

creating doc/manual/R-lang.html

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

making rw-FAQ

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*/ at C:/Rtools/texinfo5/Texinfo/Parser.pm line 5441.

Unescaped left brace in regex is deprecated here (and will be fatal in Perl
5.32), passed through in regex; marked by <-- HERE in
m/^\s+@([[:alnum:]][[:alnum:]\-]*)({
<-- HERE })?\s*(\@(c|comment)((\@|\s+).*)?)?/ at
C:/Rtools/texinfo5/Texinfo/Parser.pm line 5446.

Negative repeat count does nothing at
C:/Rtools/texinfo5/Texinfo/Convert/Plaintext.pm line 1065.

Negative repeat count does nothing at
C:/Rtools/texinfo5/Texinfo/Convert/Plaintext.pm line 1065.

Negative repeat count does nothing at
C:/Rtools/texinfo5/Texinfo/Convert/Plaintext.pm line 1065.

Negative repeat count does nothing at
C:/Rtools/texinfo5/Te

Re: [Rd] nlminb with constraints failing on some platforms

[Avraham Adler]

No error on Windows 10, R.3.5.2 patched, Rblas compiled with OpenBLAS
0.20, Rlapack is base.

> f <- function(x) sum( log(diff(x)^2+.01) + (x[1]-1)^2 )
> opt <- nlminb(rep(0, 10), f, lower=-1, upper=3)
> str(opt)
List of 6
 $ par: num [1:10] 1 1 1 1 1 ...
 $ objective  : num -41.4
 $ convergence: int 0
 $ iterations : int 66
 $ evaluations: Named int [1:2] 96 830
  ..- attr(*, "names")= chr [1:2] "function" "gradient"
 $ message: chr "relative convergence (4)"
> xhat <- rep(1, 10)
> all.equal(opt$par, xhat,  tol=0)
[1] "Mean relative difference: 3.266165e-07"
> all.equal(opt$objective, f(xhat), tol=0)
[1] "Mean relative difference: 6.722005e-13"
> abs( opt$objective - f(xhat) ) < 1e-4
[1] TRUE

> sessionInfo()
R version 3.5.2 Patched (2018-12-26 r75909)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.5.2 tools_3.5.2yaml_2.2.0

On Fri, Feb 1, 2019 at 3:24 PM Rui Barradas  wrote:
>
> Hello,
>
> R 3.5.2 on ubuntu 18.04. sessionInfo() at the end.
> Works with me, same results, cannot reproduce the error.
>
>
> f <- function(x) sum( log(diff(x)^2+.01) + (x[1]-1)^2 )
> opt <- nlminb(rep(0, 10), f, lower=-1, upper=3)
> str(opt)
>
> xhat <- rep(1, 10)
> all.equal(opt$par, xhat,  tol=0) # good: 5.53 e-7
> #[1] "Mean relative difference: 5.534757e-07"
> all.equal(opt$objective, f(xhat), tol=0) # good: 1.8 e-12
> #[1] "Mean relative difference: 1.816536e-12"
> abs( opt$objective - f(xhat) ) < 1e-4  ## Must be TRUE
> #[1] TRUE
>
>
> Hope this helps,
>
> Rui Barradas
>
>
> sessionInfo()
> R version 3.5.2 (2018-12-20)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 18.04.1 LTS
>
> Matrix products: default
> BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
>
> locale:
>   [1] LC_CTYPE=pt_PT.UTF-8   LC_NUMERIC=C
>   [3] LC_TIME=pt_PT.UTF-8LC_COLLATE=pt_PT.UTF-8
>   [5] LC_MONETARY=pt_PT.UTF-8LC_MESSAGES=pt_PT.UTF-8
>   [7] LC_PAPER=pt_PT.UTF-8   LC_NAME=C
>   [9] LC_ADDRESS=C   LC_TELEPHONE=C
> [11] LC_MEASUREMENT=pt_PT.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats graphics  grDevices utils datasets  methods   base
>
> loaded via a namespace (and not attached):
>   [1] Rcpp_1.0.0   rstudioapi_0.8   bindr_0.1.1  magrittr_1.5
>   [5] tidyselect_0.2.5 munsell_0.5.0colorspace_1.3-2 lattice_0.20-38
>   [9] R6_2.3.0 rlang_0.3.0.1stringr_1.3.1plyr_1.8.4
> [13] dplyr_0.7.8  tools_3.5.2  grid_3.5.2   yaml_2.2.0
> [17] assertthat_0.2.0 tibble_1.4.2 crayon_1.3.4 bindrcpp_0.2.2
> [21] purrr_0.2.5  reshape2_1.4.3   glue_1.3.0   stringi_1.2.4
> [25] compiler_3.5.2   pillar_1.3.1 scales_1.0.0 lubridate_1.7.4
> [29] pkgconfig_2.0.2  zoo_1.8-4
>
>
>
>
> Às 09:00 de 01/02/2019, Martin Maechler escreveu:
> >> Kasper Kristensen via R-devel
> >>  on Mon, 28 Jan 2019 08:56:39 + writes:
> >
> >  > I've noticed unstable behavior of nlminb on some Linux
> >  > systems. The problem can be reproduced by compiling
> >  > R-3.5.2 using gcc-8.2 and running the following snippet:
> >
> >  > f <- function(x) sum( log(diff(x)^2+.01) + (x[1]-1)^2 )
> >  > opt <- nlminb(rep(0, 10), f, lower=-1, upper=3)
> >  > xhat <- rep(1, 10)
> >  > abs( opt$objective - f(xhat) ) < 1e-4  ## Must be TRUE
> >
> >  > The example works perfectly when removing the bounds. However, when 
> > bounds are added the snippet returns 'FALSE'.
> >
> >  > An older R version (3.4.4), compiled using the same gcc-8.2, did not 
> > have the problem. Between the two versions R has changed the flags to 
> > compile Fortran sources:
> >
> >  > < SAFE_FFLAGS = -O2 -fomit-frame-pointer -ffloat-store
> >  > ---
> >  >> SAFE_FFLAGS = -O2 -fomit-frame-pointer -msse2 -mfpmath=sse
> >
> >  > Reverting to the old SAFE_FFLAGS 'solves' the problem.
> >
> >  >> sessionInfo()
> >  > R version 3.5.2 (2018-12-20)
> >  > Platform: x86_64-pc-linux-gnu (64-bit)
> >  > Running under: Scientific Linux release 6.4 (Carbon)
> >
> >  > Matrix products: default
> >  > BLAS/LAPACK: 
> > /zdata/groups/nfsopt/intel/2018update3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so
> >
> >  > locale:
> >  > [1] C
> >
> >  > attached base packages:
> >  > [1] stats graphics  grDevices utils datasets  methods   base
> >
> >  > loaded via a namespace (and not attached):
> >  > [1] compiler_3.5.2
> >
> > So you us Intel's MKL library for BLAS/LAPA

Re: [Rd] nlminb with constraints failing on some platforms

[Avraham Adler]

If it helps, the BLAS I used is compiled to use 6 threads.

On Wed, Feb 6, 2019 at 3:47 AM Berend Hasselman  wrote:

>
>
> > On 6 Feb 2019, at 10:58, Martin Maechler 
> wrote:
> >
>
> .
> >
> ---
> >
> > I summarize what has been reported till:
> >
> > Failure in these cases
> > 
> > 1. Kasper K ("Scientific Linux", self compiled R, using Intel's MKL
> >   for BLAS/LAPACK)
> > 2. (By Bill Dunlap): Microsoft R Open (MRO) 3.4.2, also using
> >   MKL with 12 cores
> > 3. (By Brad Bell)  : R 3.5.2 Fedora 28 (x86_64) pkg, OpenBLAS(?)
> > 4. (by MM) : R 3.5.2 Fedora 28 (x86_64) pkg, BLAS+Lapack =
> OpenBLAS
> >
> > Success
> > ===
> >
> > - (by MM): R-devel, R 3.5.2 patched on FC28, *self compiled* gcc
> 8.2,
> >using R's BLAS/Lapack
> > - (by Ralf Stubner): R 3.5.2 from Debian Stable (gcc 6.2) + OpenBLAS
> > - (by Berend H.)   : R 3.5.2 [from CRAN] on macOS 10.14.3 (BLAS/Lapack
> ??)
>
> R 3.5.2 from CRAN using R's BLAS/Lapack.
>
> Berend
>
> 
>
> > It would be great if this could be solved...
> >
> > Martin
> >
> >
> >
> >> I have tried passing in the gradient and turning on the trace and it
> gives nearly the exact same trace with and without the gradient.
> >[...]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Problem with compiling OpenBLAS to work with R

[Avraham Adler]

I believe that repo just follows the directions on my blog. Without seeing
Dr. Hodges’s code, my initial concern is the many references to Cygwin. My
method specifically does not use Cygwin but MSYS2 and Mingw64/Rtools35.
That will likely change to solely Rtools40 once R3.6 is released due to the
Msys system being built in to it.

There may be some library conflicts between Cygwin and msys2/mingw64. If
possible, my suggestion would be uninstall everything and then just install
msys2 (and add in make after you to the first msys update) and rtools35.
Then there should be no conflicting libraries.

Thanks,

Avi

On Thu, Feb 28, 2019 at 12:11 AM Kenny Bell  wrote:

> This person has had apparent success - you could follow what they did or
> just download their product (with appropriate caution downloading a random
> .exe).
>
> https://github.com/thequackdaddy/R-OpenBLAS
>
> On Thu, Feb 28, 2019 at 6:28 AM Erin Hodgess 
> wrote:
>
> > Hello!
> >
> > I'm not sure if this is the right place to post this, so apologies
> > in advance if I'm not in the right list.
> >
> > I downloaded the OpenBLAS and am following Avraham Adler's great
> > instructions.  However, when I run make, things go well to a certain
> point,
> > and then go bad:
> >
> > make
> > [snip]
> >
> > touch cygopenblas_haswellp-r0.3.5.a
> > make -j 1 -C test all
> > make[1]: Entering directory
> > '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> > gfortran -O2 -Wall -frecursive -m64 -mavx2   -o sblat1 sblat1.o
> > ../cygopenblas_haswellp-r0.3.5.a  -L/usr/lib/gcc/x86_64-pc-msys/7.3.0
> > -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib/../lib
> > -L/usr/lib/../lib
> > -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib
> > -L/usr/lib/w32api  -lmsys-2.0
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001088.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_destroy'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001090.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_init'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001091.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_lock'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001094.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_trylock'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001095.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_unlock'
> > collect2.exe: error: ld returned 1 exit status
> > make[1]: *** [Makefile:134: sblat1] Error 1
> > make[1]: Leaving directory
> > '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> > make: *** [Makefile:124: tests] Error 2
> >
> >
> > I think it has something to do with the threads/pthreads but am not sure
> > how to fix it.  Any suggestions much appreciated.
> >
> > Thanks,
> > Sincerely,
> > Erin
> >
> > Erin Hodgess, PhD
> > mailto: erinm.hodg...@gmail.com
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Package inclusion in R core implementation

[Avraham Adler]

On Mon, Mar 4, 2019 at 5:01 PM J C Nash  wrote:

> As the original coder (in mid 1970s) of BFGS, CG and Nelder-Mead in
> optim(), I've
> been pushing for some time for their deprecation. They aren't "bad", but
> we have
> better tools, and they are in CRAN packages. Similarly, I believe other
> optimization
> tools in the core (optim::L-BFGS-B, nlm, nlminb) can and should be moved to
> packages (there are already 2 versions at least of LBFGS that I and Matt
> Fidler
> are merging). And optim::SANN does not match any usual expectations of
> users.
>
> I'm sure there are other tools for other tasks that can and should move to
> packages
> to streamline the work of our core team. However, I can understand that
> there is this
> awkward issue of actually doing this. I know I'm willing to help with
> preparing
> "Transition Guide" documentation and scripts, and would be surprised if
> there are
> not others. R already has a policy of full support only for current
> version, so
> hanging on to antique tools (the three codes at the top are based on
> papers all
> of which now qualify for >50 years old) seems inconsistent with other
> messages.
>
> For information: I'm coordinating a project to build understanding of what
> older algorithms are in R as the histoRicalg project. See
> https://gitlab.com/nashjc/histoRicalg. We welcome participation.
>
> Best, JN
>
> On 2019-03-04 7:59 a.m., Jim Hester wrote:
> > Conversely, what is the process to remove a package from core R? It seems
> > to me some (many?) of the packages included are there more out of
> > historical accident rather than any technical need to be in the core
> > distribution. Having them as a core (or recommended) package makes them
> > harder update independently to R and makes testing, development and
> > contribution more cumbersome.
> >
> > On Fri, Mar 1, 2019 at 4:35 AM Morgan Morgan 
> > wrote:
> >
> >> Hi,
> >>
> >> It sometimes happens that some packages get included to R like for
> example
> >> the parallel package.
> >>
> >> I was wondering if there is a process to decide whether or not to
> include a
> >> package in the core implementation of R?
> >>
> >> For example, why not include the Rcpp package, which became for a lot of
> >> user the main tool to extend R?
> >>
> >> What is our view on the (not so well known) dotCall64 package which is
> an
> >> interesting alternative for extending R?
> >>
> >> Thank you
> >> Best regards,
> >> Morgan
> >>
>

I have No arguments with updating code to more correct or modern versions,
but I think that as a design decision, base R should have optimization
routines as opposed to it being an external package which conceptually
could be orphaned. Or at least some package gets made recommended and
adopted by R core.

Thank you,

Avi
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Inno Setup 6.0.2 fails before creating exe file on Windows (R-3.6.0)

[Avraham Adler]

I am working on compiling R-3.6.0 for Windows 10 64bit using rtools40
(beta 11). I had also installed the most recent update of Inno setup,
which is now 6.0.2.With that version, `make risntaller` fails at the
call to ""C:/R/Inno/iscc" R.iss > R-3.6.0.log 2>&1" and just exits,
pointing to line 175 of the makefile which is:

$(RPREFIX)-win.exe: R.iss
"$(ISDIR)/iscc" R.iss > $(RPREFIX).log 2>&1

Reinstalling Inno Setup 5.6.1 does allow the exe file to be created.

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Fw: Calling a LAPACK subroutine from R

[Avraham Adler]

Can you write a small C function that calls LAPACK call that fro your
Fortran code? Yes, an extra step but maybe less traumatic than rewriting
parts of LAPACK directly.

Avi

On Wed, Sep 11, 2019 at 4:08 PM Göran Broström 
wrote:

> Berend,
>
> I do not think this works with gfortran 7+. I am calling the BLAS
> subroutine dgemv from Fortran code in my package eha, and the check
> (with R-devel) gives:
>
> gmlfun.f:223:1: warning: type of ‘dgemv’ does not match original
> declaration [-Wlto-type-mismatch]
>& score, ione)
>   ^
> /home/gobr0002/R/src/R-devel/include/R_ext/BLAS.h:107:1: note: type
> mismatch in parameter 12
>   F77_NAME(dgemv)(const char *trans, const int *m, const int *n,
>
> Type of a Fortran subroutine is matched against type of a C function?!
> My conclusion is that it is impossible to call a BLAS subroutine with a
> character parameter from Fortran code (nowadays). Calling from C code is
> fine, on the other hand(!).
>
> I have recently asked about this on R-pkg-devel, but not received any
> useful answers, and my submission to CRAN is rejected. I solve it by
> making a personal copy of dgemv and changing the character parameter to
> integer, and adding Jack Dongarra, Jeremy Du Croz, Sven Hammarling, and
> Richard Hanson as authors of eha. And a Copyright note, all in the
> DESCRIPTION file. Ugly but what can I do (except rewriting the Fortran
> code in C with f2c)?
>
> Göran
>
> On 2019-09-11 21:38, Berend Hasselman wrote:
> >
> > The Lapack library is loaded automatically by R itself when it needs it
> for doing some calculation.
> > You can force it to do that with a (dummy) solve for example.
> > Put this at start of your script:
> >
> > 
> > # dummy code to get LAPACK library loaded
> > X1 <- diag(2,2)
> > x1 <- rep(2,2)
> > # X1;x1
> > z <- solve(X1,x1)
> > 
> >
> > followed by the rest of your script.
> > You will get a warning (I do) that  "passing a character vector  to
> .Fortran is not portable".
> > On other systems this may gave fatal errors. This is quick and very
> dirty. Don't do it.
> >
> > I believe there is a better and much safer way to achieve what you want.
> > Here goes.
> >
> > Create a folder (directory) src in the directory where your script
> resides.
> > Create a wrapper for "dpbtrf" file in a file xdpbtrf.f that takes an
> integer instead of character
> >
> > 
> > c intermediate for dpbtrf
> >
> >SUBROUTINE xDPBTRF( kUPLO, N, KD, AB, LDAB, INFO )
> >
> > c  .. Scalar Arguments ..
> >integer kUPLO
> >INTEGER INFO, KD, LDAB, N
> >
> > c  .. Array Arguments ..
> >DOUBLE PRECISION   AB( LDAB, * )
> >
> >character UPLO
> > c convert integer argument to character
> >if(kUPLO .eq. 1 ) then
> >UPLO = 'L'
> >else
> >UPLO = 'U'
> >endif
> >
> >call dpbtrf(UPLO,N,KD,AB,LDAB,INFO)
> >return
> >end
> > 
> >
> >
> > Instead of a character argument UPLO it takes an integer argument kUPLO.
> > The meaning should be obvious from the code.
> >
> > Now create a shell script in the folder of your script to generate a
> dynamic library to be loaded in your script:
> >
> > 
> > # Build a binary dynamic library for accessing Lapack dpbtrf
> >
> > # syntax checking
> >
> > SONAME=xdpbtrf.so
> >
> > echo Strict syntax checking
> > echo --
> > gfortran -c -fsyntax-only -fimplicit-none -Wall src/*.f || exit 1
> >
> > LAPACK=$(R CMD config LAPACK_LIBS)
> > R CMD SHLIB --output=${SONAME} src/*.f ${LAPACK} || exit 1
> > 
> >
> > To load the dynamic library xdpbtrf.so  change your script into this
> >
> > 
> > dyn.load("xdpbtrf.so")
> > n <- 4L
> > phi <- 0.64
> > AB <- matrix(0, 2, n)
> > AB[1, ] <- c(1, rep(1 + phi^2, n-2), 1)
> > AB[2, -n] <- -phi
> > round(AB, 3)
> >
> > AB.ch <- .Fortran("xdpbtrf", kUPLO=1L, N = as.integer(n),
> >  KD = 1L, AB = AB, LDAB = 2L, INFO =
> as.integer(0))$AB
> > AB.ch
> >
> > 
> >
> > and you are good to go.
> >
> > You should always do something  as described above when you need to pass
> character arguments to Fortran code.
> >
> > All of this was tested and run on macOS using the CRAN version of R.
> >
> > Berend Hasselman
> >
> >> On 11 Sep 2019, at 15:47, Giovanni Petris  wrote:
> >>
> >> Sorry for cross-posting, but I realized my question might be more
> appropriate for r-devel...
> >>
> >> Thank you,
> >> Giovanni
> >>
> >> 
> >> From: R-help  on behalf of Giovanni
> Petris 
> >> Sent: Tuesday, September 10, 2019 16:44
> >> To: r-h...@r-project.org
> >> Subject: [R] Calling a LAPACK subroutine from R
> >>
> >> Hello R-helpers!
> >>
> >> I am trying to call a LAPACK subroutine directly from my R code using
> .Fortran(), but R cannot find the symbol name. How can I register/load the
> appropriate library?
> >>
> >>> ### AR(1) Precision matrix
> >>> n <- 4L
> >>> phi <- 0.64
> >>> AB <- matrix(0, 2, n)
> >>> AB[1, ] <- c

Re: [Rd] Fw: Calling a LAPACK subroutine from R

[Avraham Adler]

I think better stated that it is subset that relied on a “bug” that was
never trapped for until now. We’re it written “properly” this never would
have arisen. At least to the best of my understanding.

Avi

On Wed, Sep 11, 2019 at 4:34 PM Göran Broström 
wrote:

>
>
> On 2019-09-11 22:16, Avraham Adler wrote:
> > Can you write a small C function that calls LAPACK call that fro your
> > Fortran code? Yes, an extra step but maybe less traumatic than rewriting
> > parts of LAPACK directly.
>
> Yes, I know how to do that, but I find it somewhat bizarre that it is
> impossible to call a Fortran subroutine from Fortran. And rewriting
> 'dgemv' was simple: Just change character to integer and 'N' to 1. And
> rename the subroutine. The hard (tedious) part was to include all the
> LAPACK authors in my DESCRIPTION file.
>
> My guess is that the root cause is that BLAS/LAPACK is written in
> FORTRAN 77, which is said to be a subset of the current Fortran version
> but obviously isn't.
>
> Thanks, Göran
>
> >
> > Avi
> >
> > On Wed, Sep 11, 2019 at 4:08 PM Göran Broström  > <mailto:goran.brost...@umu.se>> wrote:
> >
> > Berend,
> >
> > I do not think this works with gfortran 7+. I am calling the BLAS
> > subroutine dgemv from Fortran code in my package eha, and the check
> > (with R-devel) gives:
> >
> > gmlfun.f:223:1: warning: type of ‘dgemv’ does not match original
> > declaration [-Wlto-type-mismatch]
> > & score, ione)
> >^
> > /home/gobr0002/R/src/R-devel/include/R_ext/BLAS.h:107:1: note: type
> > mismatch in parameter 12
> >F77_NAME(dgemv)(const char *trans, const int *m, const int *n,
> >
> > Type of a Fortran subroutine is matched against type of a C
> function?!
> > My conclusion is that it is impossible to call a BLAS subroutine
> with a
> > character parameter from Fortran code (nowadays). Calling from C
> > code is
> > fine, on the other hand(!).
> >
> > I have recently asked about this on R-pkg-devel, but not received any
> > useful answers, and my submission to CRAN is rejected. I solve it by
> > making a personal copy of dgemv and changing the character parameter
> to
> > integer, and adding Jack Dongarra, Jeremy Du Croz, Sven Hammarling,
> and
> > Richard Hanson as authors of eha. And a Copyright note, all in the
> > DESCRIPTION file. Ugly but what can I do (except rewriting the
> Fortran
> > code in C with f2c)?
> >
> > Göran
> >
> > On 2019-09-11 21:38, Berend Hasselman wrote:
> >  >
> >  > The Lapack library is loaded automatically by R itself when it
> > needs it  for doing some calculation.
> >  > You can force it to do that with a (dummy) solve for example.
> >  > Put this at start of your script:
> >  >
> >  > 
> >  > # dummy code to get LAPACK library loaded
> >  > X1 <- diag(2,2)
> >  > x1 <- rep(2,2)
> >  > # X1;x1
> >  > z <- solve(X1,x1)
> >  > 
> >  >
> >  > followed by the rest of your script.
> >  > You will get a warning (I do) that  "passing a character vector
> > to .Fortran is not portable".
> >  > On other systems this may gave fatal errors. This is quick and
> > very dirty. Don't do it.
> >  >
> >  > I believe there is a better and much safer way to achieve what
> > you want.
> >  > Here goes.
> >  >
> >  > Create a folder (directory) src in the directory where your
> > script resides.
> >  > Create a wrapper for "dpbtrf" file in a file xdpbtrf.f that takes
> > an integer instead of character
> >  >
> >  > 
> >  > c intermediate for dpbtrf
> >  >
> >  >SUBROUTINE xDPBTRF( kUPLO, N, KD, AB, LDAB, INFO )
> >  >
> >  > c  .. Scalar Arguments ..
> >  >integer kUPLO
> >  >INTEGER INFO, KD, LDAB, N
> >  >
> >  > c  .. Array Arguments ..
> >  >DOUBLE PRECISION   AB( LDAB, * )
> >  >
> >  >character UPLO
> >  > c convert integer argument to character
> >  >if(kUPLO .eq. 1 ) then
> >  >UPLO = 'L'
> >  >else
> >  > 

Re: [Rd] New matrix function

[Avraham Adler]

It’s rather difficult. For example, the base R Kendall tau is written with
the naive O(n^2). The much faster O(n log n) implementation was programmed
and is in the pcaPP package. When I say much faster, I mean that my
implementation in Excel VBA was faster than R for 10,000 or so pairs.
R-Core decided not to implement that code, and instead made a note about
the faster implementation living in pcaPP in the help for “cor”. See [1]
for the 2012 discussion. My point is it’s really really difficult to get
something in Base R. Develop it well, put it in a package, and you have
basically the same result.

Avi

[1] https://stat.ethz.ch/pipermail/r-devel/2012-June/064351.html

On Fri, Oct 11, 2019 at 9:55 AM Morgan Morgan 
wrote:

> How do you prove usefulness of a feature?
> Do you have an example of a feature that has been added after proving to be
> useful in the package space first?
>
> Thank you,
> Morgan
>
> On Fri, 11 Oct 2019 13:53 Michael Lawrence, 
> wrote:
>
> > Thanks for this interesting suggestion, Morgan. While there is no strict
> > criteria for base R inclusion, one criterion relevant in this case is
> that
> > the usefulness of a feature be proven in the package space first.
> >
> > Michael
> >
> >
> > On Fri, Oct 11, 2019 at 5:19 AM Morgan Morgan  >
> > wrote:
> >
> >> On Fri, 11 Oct 2019 10:45 Duncan Murdoch, 
> >> wrote:
> >>
> >> > On 11/10/2019 6:44 a.m., Morgan Morgan wrote:
> >> > > Hi All,
> >> > >
> >> > > I was looking for a function to find a small matrix inside a larger
> >> > matrix
> >> > > in R similar to the one described in the following link:
> >> > >
> >> > >
> >> >
> >>
> https://www.mathworks.com/matlabcentral/answers/194708-index-a-small-matrix-in-a-larger-matrix
> >> > >
> >> > > I couldn't find anything.
> >> > >
> >> > > The above function can be seen as a "generalisation" of the "which"
> >> > > function as well as the function described in the following post:
> >> > >
> >> > >
> >> >
> >>
> https://coolbutuseless.github.io/2018/04/03/finding-a-length-n-needle-in-a-haystack/
> >> > >
> >> > > Would be possible to add such a function to base R?
> >> > >
> >> > > I am happy to work with someone from the R core team (if you wish)
> and
> >> > > suggest an implementation in C.
> >> >
> >> > That seems like it would sometimes be a useful function, and maybe
> >> > someone will point out a package that already contains it.  But if
> not,
> >> > why would it belong in base R?
> >> >
> >>
> >> If someone already implemented it, that would great indeed. I think it
> is
> >> a
> >> very general and basic function, hence base R could be a good place for
> >> it?
> >>
> >> But this is probably not a good reason; maybe someone from the R core
> team
> >> can shed some light on how they decide whether or not to include a
> >> function
> >> in base R?
> >>
> >>
> >> > Duncan Murdoch
> >> >
> >>
> >> [[alternative HTML version deleted]]
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
> >
> >
> > --
> > Michael Lawrence
> > Scientist, Bioinformatics and Computational Biology
> > Genentech, A Member of the Roche Group
> > Office +1 (650) 225-7760
> > micha...@gene.com
> >
> > Join Genentech on LinkedIn | Twitter | Facebook | Instagram | YouTube
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] New matrix function

[Avraham Adler]

Perhaps. But aren’t you looking to implementation a function which finds a
submatrix? If I’m confused, please accept my apologies.

Avi

On Fri, Oct 11, 2019 at 10:43 AM Morgan Morgan 
wrote:

> I think you are confusing package and function here. Plus some of the R
> Core packages, that you mention, contain functions that should probably be
> replaced by functions with better implementation from packages on CRAN.
>
> Best regards
> Morgan
>
> On Fri, 11 Oct 2019 15:22 Joris Meys,  wrote:
>
> >
> >
> > On Fri, Oct 11, 2019 at 3:55 PM Morgan Morgan  >
> > wrote:
> >
> >> How do you prove usefulness of a feature?
> >> Do you have an example of a feature that has been added after proving to
> >> be
> >> useful in the package space first?
> >>
> >> Thank you,
> >> Morgan
> >>
> >
> > The parallel package (a base package like utils, stats, ...) was added as
> > a drop-in replacement of the packages snow and multicore for parallel
> > computing. That's one example, but sure there's more.
> >
> > Kind regards
> > Joris
> >
> > --
> > Joris Meys
> > Statistical consultant
> >
> > Department of Data Analysis and Mathematical Modelling
> > Ghent University
> > Coupure Links 653, B-9000 Gent (Belgium)
> >
> > <
> https://maps.google.com/?q=Coupure+links+653,%C2%A0B-9000+Gent,%C2%A0Belgium&entry=gmail&source=g
> >
> >
> > ---
> > Biowiskundedagen 2018-2019
> > http://www.biowiskundedagen.ugent.be/
> >
> > ---
> > Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] improving the performance of install.packages

[Avraham Adler]

Exactly. Every major commit isn’t want to check that the package works.

Also, besides package development, there are other reasons why one would
install packages over themselves. For example, rebuilding from source after
changing options in Makevars[.win]. The package hasn’t been updated but
recompilation is desired.

Avi

On Fri, Nov 8, 2019 at 3:07 PM William Dunlap via R-devel <
r-devel@r-project.org> wrote:

> While developing a package, I often run install.packages() on it many times
> in a session without updating its version number.  How would your proposed
> change affect this workflow?
> Bill Dunlap
> TIBCO Software
> wdunlap tibco.com
>
>
> On Fri, Nov 8, 2019 at 11:56 AM Joshua Bradley 
> wrote:
>
> > I could do this...and I have before. This brings up a more fundamental
> > question though. You're asking me to write code that changes the logic of
> > the installation process (i.e. writing my own package installer). Instead
> > of doing that, I would rather integrate that logic into R itself to
> improve
> > the baseline installation process. This api proposal change would be
> > additive and would not break legacy code.
> >
> > Package managers like pip (python), conda (python), yum (CentOS), apt
> > (Ubuntu), and apk (Alpine) are all "smart" enough to know (by their
> > defaults) when to not download a package again. By proposing this change,
> > I'm essentially asking that R follow some of the same conventions and
> best
> > practices that other package managers have adopted over the decades.
> >
> > I assumed this list is used to discuss proposals like this to the R
> > codebase. If I'm on the wrong list, please let me know.
> >
> > P.S. if this change happened, it would be interesting to study the effect
> > it has on the bandwidth across all CRAN mirrors. A significant drop would
> > turn into actual $$ saved
> >
> > Josh Bradley
> >
> >
> > On Fri, Nov 8, 2019 at 5:00 AM Duncan Murdoch 
> > wrote:
> >
> > > On 08/11/2019 2:06 a.m., Joshua Bradley wrote:
> > > > Hello,
> > > >
> > > > Currently if you install a package twice:
> > > >
> > > > install.packages("testit")
> > > > install.packages("testit")
> > > >
> > > > R will build the package from source (depending on what OS you're
> > using)
> > > > twice by default. This becomes especially burdensome when people are
> > > using
> > > > big packages (i.e. lots of depends) and someone has a script with:
> > > >
> > > > install.packages("tidyverse")
> > > > ...
> > > > ... later on down the script
> > > > ...
> > > > install.packages("dplyr")
> > > >
> > > > In this case, "dplyr" is part of the tidyverse and will install
> twice.
> > As
> > > > the primary "package manager" for R, it should not install a package
> > > twice
> > > > (by default) when it can be so easily checked. Indeed, many people
> > resort
> > > > to writing a few lines of code to filter out already-installed
> packages
> > > An
> > > > r-help post from 2010 proposed a solution to improving the default
> > > > behavior, by adding "force=FALSE" as a api addition to
> > install.packages.(
> > > > https://stat.ethz.ch/pipermail/r-help/2010-May/239492.html)
> > > >
> > > > Would the R-core devs still consider this proposal?
> > >
> > > Whether or not they'd do it, it's easy for you to do it.
> > >
> > > install.packages <- function(pkgs, ..., force = FALSE) {
> > >if (!force) {
> > >  pkgs <- Filter(Negate(requireNamespace), pkgs
> > >
> > >utils::install.packages(pkgs, ...)
> > > }
> > >
> > > You might want to make this more elaborate, e.g. doing
> update.packages()
> > > on the ones that exist.  But really, isn't the problem with the script
> > > you're using, which could have done a simple test before forcing a slow
> > > install?
> > >
> > > Duncan Murdoch
> > >
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] switch to reference counting in R-devel

[Avraham Adler]

Agreed. Now is as good a time as any to send many,  many thanks are due to
Luke, Martin, Uwe, Duncan, the redoubtable  Professor B. and the entire
R-Core team for their seemingly countless hours of toil keeping R not only
afloat but healthy and vibrant. Your work is deeply appreciated, even if it
isn’t expressed often enough.

Thank you again!

Avi

On Tue, Dec 3, 2019 at 1:11 PM Henrik Bengtsson 
wrote:

> This is very exciting news.  Luke, thank you for all your work on this
> - I know it's been a long journey.
>
> All the best,
>
> Henrik
>
> On Tue, Dec 3, 2019 at 8:04 AM Tierney, Luke 
> wrote:
> >
> > R-devel has been switched to use reference counting by default with
> > r77508. Building with -DSWITCH_TO_NAMED goes back to the NAMED
> > mechanism.
> >
> > Best,
> >
> > luke
> >
> > On Sun, 24 Nov 2019, luke-tier...@uiowa.edu wrote:
> >
> > > Baring any unforeseen issues R-devel will switch in about a week from
> > > the NAMED mechanism to reference counting for determining when objects
> > > can be safely mutated in base C code. This is expected to have minimal
> > > impact on packages not using unsupported coding practices in their C
> > > code.
> > >
> > >
> > > The transition to reference counting has been in progress for a
> > > number of years. Some older notes on this are available at
> > > http://developer.r-project.org/Refcnt.html.  These may no longer be
> > > completely accurate but should give you an idea of what is going on.
> > >
> > > If you want to test your package under reference counting you can do
> > > so by building R with -DSWITCH_TO_REFCNT added to CFLAGS or DEFS in a
> > > config.site file.
> > >
> > > A small number of packages are still using the NAMED or SET_NAMED
> > > functions even though this has been discouraged for some  time.
> > > For now these will not produce errors but also not do anything useful.
> > > They will probably be removed before R 4.0.0 is released, so you
> > > should look at why you are using them and adjust accordingly.
> > >
> > > Best,
> > >
> > > luke
> > >
> > >
> > >
> >
> > --
> > Luke Tierney
> > Ralph E. Wareham Professor of Mathematical Sciences
> > University of Iowa  Phone: 319-335-3386
> > Department of Statistics andFax:   319-335-3017
> > Actuarial Science
> > 241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
> > Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R 3.6.2 is released

[Avraham Adler]

Hi.

Under R-news there is an entry for 3.6.2 patched regarding LAPACK. However,
when uncompresding the current R-patched, it creates R-Rc directories. Is
this a naming oversight or is the patched version actually the unadjusted
release candidate?

Thank you,

Avi

On Thu, Dec 12, 2019 at 4:58 AM Peter Dalgaard via R-devel <
r-devel@r-project.org> wrote:

> The build system rolled up R-3.6.2.tar.gz (codename "Dark and Stormy
> Night") this morning.
>
> The list below details the changes in this release.
>
> You can get the source code from
>
> http://cran.r-project.org/src/base/R-3/R-3.6.2.tar.gz
>
> or wait for it to be mirrored at a CRAN site nearer to you.
>
> Binaries for various platforms will appear in due course.
>
>
> For the R Core Team,
>
> Peter Dalgaard
>
> These are the checksums (md5 and SHA-256) for the freshly created files,
> in case you wish
> to check that they are uncorrupted:
>
> MD5 (AUTHORS) = b9c44f9f78cab3184ad9898bebc854b4
> MD5 (COPYING) = eb723b61539feef013de476e68b5c50a
> MD5 (COPYING.LIB) = a6f89e2100d9b6cdffcea4f398e37343
> MD5 (FAQ) = 28a3942a7129877e9af1d5ea16202052
> MD5 (INSTALL) = 7893f754308ca31f1ccf62055090ad7b
> MD5 (NEWS) = 45437b38c75e0248b527c00e6d42ee6a
> MD5 (NEWS.0) = bfcd7c147251b5474d96848c6f57e5a8
> MD5 (NEWS.1) = eb78c4d053ec9c32b815cf0c2ebea801
> MD5 (NEWS.2) = 591dcf615162127f904e4e461f330ce9
> MD5 (R-latest.tar.gz) = 90d23d138cee26d275da14b58296e521
> MD5 (README) = f468f281c919665e276a1b691decbbe6
> MD5 (RESOURCES) = 529223fd3ffef95731d0a87353108435
> MD5 (THANKS) = bb45f89c01d509721c47fd41f147da60
> MD5 (VERSION-INFO.dcf) = 9c33701e25092aefc1d16beb5858f20f
> MD5 (R-3/R-3.6.2.tar.gz) = 90d23d138cee26d275da14b58296e521
>
>
> 2cde824a7b18958e5f06b391c801c8288be0f84fa8934b7ddefef23c67e60c09  AUTHORS
> e6d6a009505e345fe949e1310334fcb0747f28dae2856759de102ab66b722cb4  COPYING
> 6095e9ffa777dd22839f7801aa845b31c9ed07f3d6bf8a26dc5d2dec8ccc0ef3
> COPYING.LIB
> 38219d9c6221ccfbf075ef03711b420a1aa8731f890c8f2337148b602a217c2d  FAQ
> f87461be6cbaecc4dce44ac58e5bd52364b0491ccdadaf846cb9b452e9550f31  INSTALL
> 0ceb6fbab3e0e29bc374683fd5c2ccd0c9c62ce8eca2a394a4603775b3ef129c  NEWS
> 4e21b62f515b749f80997063fceab626d7258c7d650e81a662ba8e0640f12f62  NEWS.0
> 12b30c724117b1b2b11484673906a6dcd48a361f69fc420b36194f9218692d01  NEWS.1
> ca04f78ffe54afa326fe3ed40e7e1411acaed2fa5ead97ddf51c6aa5b7bc  NEWS.2
> bd65a45cddfb88f37370fbcee4ac8dd3f1aebeebe47c2f968fd9770ba2bbc954
> R-latest.tar.gz
> 2fdd3e90f23f32692d4b3a0c0452f2c219a10882033d1774f8cadf25886c3ddc  README
> 408737572ecc6e1135fdb2cf7a9dbb1a6cb27967c757f1771b8c39d1fd2f1ab9  RESOURCES
> 2a8dca916cd92229ef9e328f3610ca204809c262823b860252b42072dac2473a  THANKS
> 40cc7cea5f0e67cf8f2f7b25a534ae6bc53f38eae2ab2c2649a952ed37f0654a
> VERSION-INFO.dcf
> bd65a45cddfb88f37370fbcee4ac8dd3f1aebeebe47c2f968fd9770ba2bbc954
> R-3/R-3.6.2.tar.gz
>
> This is the relevant part of the NEWS file
>
> CHANGES IN R 3.6.2:
>
>   NEW FEATURES:
>
> * runmed(x, *) gains a new option na.action determining _how_ to
>   handle NaN or NA in x.
>
> * dotchart() gains new options ann, xaxt, frame.plot and log.
>
>   INSTALLATION on a UNIX-ALIKE:
>
> * Detection of the C stack direction has been moved from run-time
>   to configure: this is safer with LTO builds and allows the
>   detection to be overridden - see file config.site.
>
> * Source-code changes enable installation on platforms using gcc
>   -fno-common (the expected default for gcc 10.x).
>
>   C-LEVEL FACILITIES:
>
> * installTrChar (which is nowadays is wrapped by installChar) is
>   defined in Rinternals.h.  (Neither are part of the API.)
>
>   PACKAGE INSTALLATION:
>
> * Header Rconfig.h contains the value of FC_LEN_T deduced at
>   installation which is used by the prototypes in headers
>   R_ext/BLAS.h and R_ext/Lapack.h but to avoid extensive breakage
>   this is only exposed when USE_FC_LEN_T is defined.
>
>   If a package's C/C++ calls to BLAS/LAPACK allow for the 'hidden'
>   arguments used by most Fortran compilers to pass the lengths of
>   Fortran character arguments, define USE_FC_LEN_T and include
>   Rconfig.h (possibly _via_ R.h) before including R_ext/BLAS.h or
>   R_ext/Lapack.h.
>
> * A package with Fortran source code and perhaps C (but not C++)
>   sources can request for its shared object/DLL to be linked by the
>   Fortran compiler by including a line USE_FC_TO_LINK= in
>   src/Makevars[.win] and using $(SHLIB_OPENMP_FFLAGS) as part of
>   PKG_LIBS.
>
>   The known reason for doing so is a package which uses Fortran
>   (only) OpenMP on a platform where the Fortran OpenMP runtime is
>   incompatible with the C one (e.g. gfortran 9.x with clang).
>
>   UTILITIES:
>
> * R CMD check has a new option to mitigate checks leaving
>   files/directories in /tmp.  See the 'R Internals' manual - this
>   is part of --as-cran.
>
>   Windows:
>
> * The defau

Re: [Rd] R 3.6.2 is released

[Avraham Adler]

Thank you.

I apologize for not providing the link. [1] Under the news for R-revel
there is a single entry for R 3.6.2-patched.

The file I downloaded was [2] with a date of 2019-12-12 01:50.

Is it safe to say that 3.6.2 has the LAPACK upgrades and fixes?

Apologies in advance if iOS links the URL below. I cannot access gmail
desktop from behind my corporate firewall.

Thank you again,

Avi

[1]
https://cran.r-project.org/doc/manuals/r-devel/NEWS.html
[2]
https://stat.ethz.ch/R/daily/R-patched.tar.gz

On Thu, Dec 12, 2019 at 11:10 AM Peter Dalgaard  wrote:

> It is not obvious what it is that you are calling "R-patched", nor where
> there could be an entry for "3.6.2 patched".
>
> The prerelease/patched versions are snapshots of R-3-6-branch made at
> 00:20 so the current one will have been made before the release version run
> started at 09:00 this morning, and hence the nightly tarball will be of the
> release candidate. However it will not be called R-patched:
>
> lrwxr-xr-x  1 pd  staff29 Dec 12 00:20 R-latest.tar.gz ->
> R-rc_2019-12-06_r77555.tar.gz
>
> The current version in SVN differs only by the VERSION file. Its NEWS.Rd
> starts
>
> Peters-MacBook-Air:R pd$ more VERSION
> 3.6.2 Patched
> Peters-MacBook-Air:R pd$ more doc/NEWS.Rd
> % -*- coding: utf-8 -*-
> \newcommand{\Rlogo}{\if{html}{\figure{../../html/Rlogo.svg}{options:
> class="toplogo" alt="[R logo]"}}\if{latex}{\figure{Rlogo.pdf}{options:
> width=0.5in}}}
>
> \name{NEWS}
> \title{R News}
> \encoding{UTF-8}
>
> \section{\Rlogo CHANGES IN R 3.6.2}{
>
>   \subsection{NEW FEATURES}{
> \itemize{
>   ....
>
> and any changes for 3.6.2 patched should go above the entries for 3.6.2.
>
> - pd
>
>
> > On 12 Dec 2019, at 16:34 , Avraham Adler 
> wrote:
> >
> > Hi.
> >
> > Under R-news there is an entry for 3.6.2 patched regarding LAPACK.
> However, when uncompresding the current R-patched, it creates R-Rc
> directories. Is this a naming oversight or is the patched version actually
> the unadjusted release candidate?
> >
> > Thank you,
> >
> > Avi
> >
> > On Thu, Dec 12, 2019 at 4:58 AM Peter Dalgaard via R-devel <
> r-devel@r-project.org> wrote:
> > The build system rolled up R-3.6.2.tar.gz (codename "Dark and Stormy
> Night") this morning.
> >
> > The list below details the changes in this release.
> >
> > You can get the source code from
> >
> > http://cran.r-project.org/src/base/R-3/R-3.6.2.tar.gz
> >
> > or wait for it to be mirrored at a CRAN site nearer to you.
> >
> > Binaries for various platforms will appear in due course.
> >
> >
> > For the R Core Team,
> >
> > Peter Dalgaard
> >
> > These are the checksums (md5 and SHA-256) for the freshly created files,
> in case you wish
> > to check that they are uncorrupted:
> >
> > MD5 (AUTHORS) = b9c44f9f78cab3184ad9898bebc854b4
> > MD5 (COPYING) = eb723b61539feef013de476e68b5c50a
> > MD5 (COPYING.LIB) = a6f89e2100d9b6cdffcea4f398e37343
> > MD5 (FAQ) = 28a3942a7129877e9af1d5ea16202052
> > MD5 (INSTALL) = 7893f754308ca31f1ccf62055090ad7b
> > MD5 (NEWS) = 45437b38c75e0248b527c00e6d42ee6a
> > MD5 (NEWS.0) = bfcd7c147251b5474d96848c6f57e5a8
> > MD5 (NEWS.1) = eb78c4d053ec9c32b815cf0c2ebea801
> > MD5 (NEWS.2) = 591dcf615162127f904e4e461f330ce9
> > MD5 (R-latest.tar.gz) = 90d23d138cee26d275da14b58296e521
> > MD5 (README) = f468f281c919665e276a1b691decbbe6
> > MD5 (RESOURCES) = 529223fd3ffef95731d0a87353108435
> > MD5 (THANKS) = bb45f89c01d509721c47fd41f147da60
> > MD5 (VERSION-INFO.dcf) = 9c33701e25092aefc1d16beb5858f20f
> > MD5 (R-3/R-3.6.2.tar.gz) = 90d23d138cee26d275da14b58296e521
> >
> >
> > 2cde824a7b18958e5f06b391c801c8288be0f84fa8934b7ddefef23c67e60c09  AUTHORS
> > e6d6a009505e345fe949e1310334fcb0747f28dae2856759de102ab66b722cb4  COPYING
> > 6095e9ffa777dd22839f7801aa845b31c9ed07f3d6bf8a26dc5d2dec8ccc0ef3
> COPYING.LIB
> > 38219d9c6221ccfbf075ef03711b420a1aa8731f890c8f2337148b602a217c2d  FAQ
> > f87461be6cbaecc4dce44ac58e5bd52364b0491ccdadaf846cb9b452e9550f31  INSTALL
> > 0ceb6fbab3e0e29bc374683fd5c2ccd0c9c62ce8eca2a394a4603775b3ef129c  NEWS
> > 4e21b62f515b749f80997063fceab626d7258c7d650e81a662ba8e0640f12f62  NEWS.0
> > 12b30c724117b1b2b11484673906a6dcd48a361f69fc420b36194f9218692d01  NEWS.1
> > ca04f78ffe54afa326fe3ed40e7e1411acaed2fa5ead97ddf51c6aa5b7bc  NEWS.2
> > bd65a45cddfb88f37370fbcee4ac8dd3f1aebeebe47c2f968fd9770ba2bbc954
> R-latest.tar.gz
> > 2fdd3e90f23f32692d4b3a0c0452f2c219a10882033d1774f8cadf25886c3ddc  REA

Re: [Rd] as-cran issue ==> set _R_CHECK_LENGTH_1_* settings!

[Avraham Adler]

Those of us stuck on Windows but who attempt to develop properly are
wounded to the quick, sir!

:)

Avi

On Mon, Jan 13, 2020 at 12:24 PM Martin Maechler 
wrote:

> > Ben Bolker
> > on Mon, 13 Jan 2020 11:49:09 -0500 writes:
>
> > From R NEWS (changes in 3.6.0)
> > Experimentally, setting environment variable
> _R_CHECK_LENGTH_1_LOGIC2_
> > will lead to warnings (or errors if the variable is set to a ‘true’
> > value) when && or || encounter and use arguments of length more than
> one.
>
> Indeed,  thank you, Ben.
>
> Note (Dirk) this is not just something
>   "by Henrik (..) as he tried to convince us all to use it more"
>
> I've activated this (and the other
>   _R_CHECK_LENGTH_1_CONDITION_ ! )
> for years (maybe not many years, it just feels like it), and *EVERY TIME*
> it triggers, it's been revealing a programmeR's thinko / bug / ..,
> something where the code was clearly suboptimal and should've been
> improved.
> (Unfortunately, the bug has often been in packages, and sometimes I had to
>  disable the setting when I wanted that "buggy" package to work ..)
>
> Occasionally being puristic, let me state this:
>__
>   /--\
>   |  |
>   | Every careful R programmer should use (something like "true",|
>   | "verbose", or even package=... ) |
>   |  |
>   | export _R_CHECK_LENGTH_1_CONDITION_=true |
>   | export _R_CHECK_LENGTH_1_LOGIC2_=verbose |
>   |  |
>   | in her/his ~/.profile equivalent (*) |
>   \__/
>
>
> *) well assuming a careful R programmer would never develop on
>Windows anyway (where you need different means to set such
>environment variables).
>
>
>
> > On 2020-01-13 11:46 a.m., Therneau, Terry M., Ph.D. via R-devel
> wrote:
> >> Thanks for the feedback Dirk.   I sent my follow-up before I saw it.
> >>
> >> Looking at the source code, it appears that there is no options()
> call
> >> to turn this on. Nor does "R --help" reveal a command line option.
> >> How then does a user turn this on outside of the R CMD check
> >> envirionment, so as to chase things like this down?
> >>
> >> The fact that 1. renaming my function makes the error go away, 2. my
> >> function is just a wrapper to inherits(), and 3. its a new error in
> code
> >> that hasn't changed, all point me towards some oddity with the check
> >> function.
> >>
> >> Terry
> >>
> >>
> >> On 1/13/20 10:22 AM, Dirk Eddelbuettel wrote:
> >>>
> >>> On 13 January 2020 at 10:02, Therneau, Terry M., Ph.D. via R-devel
> wrote:
> >>> | Where can I find out (and replicate) what options as-cran turns
> on?
> >>>
> >>> See the file src/library/tools/R/check.R in the R sources, and
> grep for
> >>> as_cran which is the internal variable controlled by the --as-cran
> option
> >>>
> >>> [...]
> >>>
> >>> | The check log contains multiple instances of the lines below:
> >>> |
> >>> | < Warning message:
> >>> | < In if (ismat(kmat)) { :
> >>> | <   the condition has length > 1 and only the first element will
> be
> >>> used
> >>> |
> >>> | I don't see how the error could arise, but if I know what
> as-cran is
> >>> doing perhaps I can
> >>> | replicate it.
> >>>
> >>> This was widely discussed on this list and should also be in the
> NEWS
> >>> file.
> >>>
> >>> The change is about what the message says: the if () tests a scalar
> >>> logical,
> >>> it appears that ismat(kmat) returns more than a scalar.
> >>>
> >>> There has always been an opt-in for this to error -- cf many
> messages
> >>> by Henrik
> >>> over the years as he tried to convince us all to use it more.
> >>>
> >>>
> >>> Dirk
> >>>
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] rpois(9, 1e10)

[Avraham Adler]

Maybe there should be code for 64 bit R to use long long or the like?

On Sun, Jan 19, 2020 at 10:45 AM Spencer Graves 
wrote:

>
>
> On 2020-01-19 09:34, Benjamin Tyner wrote:
> >> 
> >> Hello, All:
> >>
> >>
> >> Consider:
> >>
> >>
> >> Browse[2]> set.seed(1)
> >> Browse[2]> rpois(9, 1e10)
> >> NAs produced[1] NA NA NA NA NA NA NA NA NA
> >>
> >>
> >> Should this happen?
> >>
> >>
> >> I think that for, say, lambda>1e6, rpois should return rnorm(.,
> >> lambda, sqrt(lambda)).
> > But need to implement carefully; rpois should always return a
> > non-negative integer, whereas rnorm always returns numeric...
> >
>
>Thanks for the reply.
>
>
>However, I think it's not acceptable to get an NA from a number
> that cannot be expressed as an integer.  Whenever a randomly generated
> number would exceed .Machine$integer.max, the choice is between
> returning NA or a non-integer numeric.  Consider:
>
>
>  > 2*.Machine$integer.max
> [1] 4294967294
>  > as.integer(2*.Machine$integer.max)
> [1] NA
> Warning message:
> NAs introduced by coercion to integer range
>
>
>I'd rather have the non-integer numeric.
>
>
>Spencer
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] rpois(9, 1e10)

[Avraham Adler]

Technically, lambda can always be numeric. It is the observations which
must be integral.

Would hitting everything larger than maxint or maxlonglong with floor or
round fundamentally change the distribution? Well, yes, but enough that it
would matter over process risk?

Avi

On Sun, Jan 19, 2020 at 11:20 AM Benjamin Tyner  wrote:

> So imagine rpois is changed, such that the storage mode of its return
> value is sometimes integer and sometimes numeric. Then imagine the case
> where lambda is itself a realization of a random variable. Do we really
> want the storage mode to inherit that randomness?
>
>
> On 1/19/20 10:47 AM, Avraham Adler wrote:
> > Maybe there should be code for 64 bit R to use long long or the like?
> >
> > On Sun, Jan 19, 2020 at 10:45 AM Spencer Graves
> > mailto:spencer.gra...@prodsyse.com>>
> wrote:
> >
> >
> >
> > On 2020-01-19 09:34, Benjamin Tyner wrote:
> > >>
> >
>  
> > >> Hello, All:
> > >>
> > >>
> > >> Consider:
> > >>
> > >>
> > >> Browse[2]> set.seed(1)
> > >> Browse[2]> rpois(9, 1e10)
> > >> NAs produced[1] NA NA NA NA NA NA NA NA NA
> > >>
> > >>
> > >> Should this happen?
> > >>
> > >>
> > >> I think that for, say, lambda>1e6, rpois should return
> > rnorm(.,
> > >> lambda, sqrt(lambda)).
> > > But need to implement carefully; rpois should always return a
> > > non-negative integer, whereas rnorm always returns numeric...
> > >
> >
> >Thanks for the reply.
> >
> >
> >However, I think it's not acceptable to get an NA from a
> > number
> > that cannot be expressed as an integer.  Whenever a randomly
> > generated
> > number would exceed .Machine$integer.max, the choice is between
> > returning NA or a non-integer numeric.  Consider:
> >
> >
> >  > 2*.Machine$integer.max
> > [1] 4294967294
> >  > as.integer(2*.Machine$integer.max)
> > [1] NA
> > Warning message:
> > NAs introduced by coercion to integer range
> >
> >
> >I'd rather have the non-integer numeric.
> >
> >
> >Spencer
> >
> > __
> > R-devel@r-project.org <mailto:R-devel@r-project.org> mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> > --
> > Sent from Gmail Mobile
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] rpois(9, 1e10)

[Avraham Adler]

Crazy thought, but being that a sum of Poissons is Poisson in the sum, can
you break your “big” simulation into the sum of a few smaller ones? Or is
the order of magnitude difference just too great?

On Sun, Jan 19, 2020 at 1:58 PM Spencer Graves 
wrote:

>   This issue arose for me in simulations to estimate confidence,
> prediction, and tolerance intervals from glm(., family=poisson) fits
> embedded in a BMA::bic.glm fit using a simulate.bic.glm function I added to
> the development version of Ecfun, available at
> "https://github.com/sbgraves237/Ecfun";
> <https://github.com/sbgraves237/Ecfun>.  This is part of a vignette I'm
> developing, available at
> "https://github.com/sbgraves237/Ecfun/blob/master/vignettes/time2nextNuclearWeaponState.Rmd";
> <https://github.com/sbgraves237/Ecfun/blob/master/vignettes/time2nextNuclearWeaponState.Rmd>.
> This includes a simulated mean of a mixture of Poissons that exceeds 2e22.
> It doesn't seem unreasonable to me to have rpois output a numerics rather
> than integers when a number simulated exceeds .Machine$integer.max.  And it
> does seem to make less sense in such cases to return NAs.
>
>
>Alternatively, might it make sense to add another argument to rpois
> to give the user the choice?  E.g., an argument "bigOutput" with (I hope)
> default = "numeric" and "NA" as a second option.  Or NA is the default, so
> no code that relied that feature of the current code would be broken by the
> change.  If someone wanted to use arbitrary precision arithmetic, they
> could write their own version of this function with "arbitraryPrecision" as
> an optional value for the "bigOutput" argument.
>
>
>   Comments?
>   Thanks,
>   Spencer Graves
>
>
>
> On 2020-01-19 10:28, Avraham Adler wrote:
>
> Technically, lambda can always be numeric. It is the observations which
> must be integral.
>
> Would hitting everything larger than maxint or maxlonglong with floor or
> round fundamentally change the distribution? Well, yes, but enough that it
> would matter over process risk?
>
> Avi
>
> On Sun, Jan 19, 2020 at 11:20 AM Benjamin Tyner  wrote:
>
>> So imagine rpois is changed, such that the storage mode of its return
>> value is sometimes integer and sometimes numeric. Then imagine the case
>> where lambda is itself a realization of a random variable. Do we really
>> want the storage mode to inherit that randomness?
>>
>>
>> On 1/19/20 10:47 AM, Avraham Adler wrote:
>> > Maybe there should be code for 64 bit R to use long long or the like?
>> >
>> > On Sun, Jan 19, 2020 at 10:45 AM Spencer Graves
>> > mailto:spencer.gra...@prodsyse.com>>
>> wrote:
>> >
>> >
>> >
>> > On 2020-01-19 09:34, Benjamin Tyner wrote:
>> > >>
>> >
>>  
>> > >> Hello, All:
>> > >>
>> > >>
>> > >> Consider:
>> > >>
>> > >>
>> > >> Browse[2]> set.seed(1)
>> > >> Browse[2]> rpois(9, 1e10)
>> > >> NAs produced[1] NA NA NA NA NA NA NA NA NA
>> > >>
>> > >>
>> > >> Should this happen?
>> > >>
>> > >>
>> > >> I think that for, say, lambda>1e6, rpois should return
>> > rnorm(.,
>> > >> lambda, sqrt(lambda)).
>> > > But need to implement carefully; rpois should always return a
>> > > non-negative integer, whereas rnorm always returns numeric...
>> > >
>> >
>> >Thanks for the reply.
>> >
>> >
>> >However, I think it's not acceptable to get an NA from a
>> > number
>> > that cannot be expressed as an integer.  Whenever a randomly
>> > generated
>> > number would exceed .Machine$integer.max, the choice is between
>> > returning NA or a non-integer numeric.  Consider:
>> >
>> >
>> >  > 2*.Machine$integer.max
>> > [1] 4294967294
>> >  > as.integer(2*.Machine$integer.max)
>> > [1] NA
>> > Warning message:
>> > NAs introduced by coercion to integer range
>> >
>> >
>> >I'd rather have the non-integer numeric.
>> >
>> >
>> >Spencer
>> >
>> > __
>> > R-devel@r-project.org <mailto:R-devel@r-project.org> mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-devel
>> >
>> > --
>> > Sent from Gmail Mobile
>>
> --
> Sent from Gmail Mobile
>
>
> --
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] rpois(9, 1e10)

[Avraham Adler]

Floor (maybe round) of non-negative numerics, though. Poisson should never
have anything after decimal.

Still think it’s worth allowing long long for R64 bit, just for purity
sake.

Avi

On Sun, Jan 19, 2020 at 4:38 PM Spencer Graves 
wrote:

>
>
> On 2020-01-19 13:01, Avraham Adler wrote:
>
> Crazy thought, but being that a sum of Poissons is Poisson in the sum, can
> you break your “big” simulation into the sum of a few smaller ones? Or is
> the order of magnitude difference just too great?
>
>
>
>   I don't perceive that as feasible.  Once I found what was generating
> NAs, it was easy to code a function to return pseudo-random numbers using
> the standard normal approximation to the Poisson for those extreme cases.
> [For a Poisson with mean = 1e6, for example, the skewness (third
> standardized moment) is 0.001.  At least for my purposes, that should be
> adequate.][1]
>
>
>   What are the negative consequences of having rpois return numerics
> that are always nonnegative?
>
>
>   Spencer
>
>
> [1]  In the code I reported before, I just changed the threshold of 1e6 to
> 0.5*.Machine$integer.max.  On my Mac, .Machine$integer.max = 2147483647 =
> 2^31 > 1e9.  That still means that a Poisson distributed pseudo-random
> number just under that would have to be over 23000 standard deviations
> above the mean to exceed .Machine$integer.max.
>
>
> On Sun, Jan 19, 2020 at 1:58 PM Spencer Graves <
> spencer.gra...@prodsyse.com> wrote:
>
>>   This issue arose for me in simulations to estimate confidence,
>> prediction, and tolerance intervals from glm(., family=poisson) fits
>> embedded in a BMA::bic.glm fit using a simulate.bic.glm function I added to
>> the development version of Ecfun, available at
>> "https://github.com/sbgraves237/Ecfun";
>> <https://github.com/sbgraves237/Ecfun>.  This is part of a vignette I'm
>> developing, available at
>> "https://github.com/sbgraves237/Ecfun/blob/master/vignettes/time2nextNuclearWeaponState.Rmd";
>> <https://github.com/sbgraves237/Ecfun/blob/master/vignettes/time2nextNuclearWeaponState.Rmd>.
>> This includes a simulated mean of a mixture of Poissons that exceeds 2e22.
>> It doesn't seem unreasonable to me to have rpois output a numerics rather
>> than integers when a number simulated exceeds .Machine$integer.max.  And it
>> does seem to make less sense in such cases to return NAs.
>>
>>
>>Alternatively, might it make sense to add another argument to
>> rpois to give the user the choice?  E.g., an argument "bigOutput" with (I
>> hope) default = "numeric" and "NA" as a second option.  Or NA is the
>> default, so no code that relied that feature of the current code would be
>> broken by the change.  If someone wanted to use arbitrary precision
>> arithmetic, they could write their own version of this function with
>> "arbitraryPrecision" as an optional value for the "bigOutput" argument.
>>
>>
>>   Comments?
>>   Thanks,
>>   Spencer Graves
>>
>>
>>
>> On 2020-01-19 10:28, Avraham Adler wrote:
>>
>> Technically, lambda can always be numeric. It is the observations which
>> must be integral.
>>
>> Would hitting everything larger than maxint or maxlonglong with floor or
>> round fundamentally change the distribution? Well, yes, but enough that it
>> would matter over process risk?
>>
>> Avi
>>
>> On Sun, Jan 19, 2020 at 11:20 AM Benjamin Tyner  wrote:
>>
>>> So imagine rpois is changed, such that the storage mode of its return
>>> value is sometimes integer and sometimes numeric. Then imagine the case
>>> where lambda is itself a realization of a random variable. Do we really
>>> want the storage mode to inherit that randomness?
>>>
>>>
>>> On 1/19/20 10:47 AM, Avraham Adler wrote:
>>> > Maybe there should be code for 64 bit R to use long long or the like?
>>> >
>>> > On Sun, Jan 19, 2020 at 10:45 AM Spencer Graves
>>> > mailto:spencer.gra...@prodsyse.com>>
>>> wrote:
>>> >
>>> >
>>> >
>>> > On 2020-01-19 09:34, Benjamin Tyner wrote:
>>> > >>
>>> >
>>>  
>>> > >> Hello, All:
>>> > >>
>>> > >>
>>> > >> Consider:
>>> > >>
>>> > >>
>>> >  

Re: [Rd] [External] Re: rpois(9, 1e10)

[Avraham Adler]

Fantastic!!

Thanks,

Avi

On Wed, Jan 22, 2020 at 11:14 AM Spencer Graves 
wrote:

>
>
> On 2020-01-22 02:54, Martin Maechler wrote:
> >> Martin Maechler
> >>  on Tue, 21 Jan 2020 09:25:19 +0100 writes:
> >> Ben Bolker
> >>  on Mon, 20 Jan 2020 12:54:52 -0500 writes:
> >  >> Ugh, sounds like competing priorities.
> >
> >  > indeed.
> >
> >  >> * maintain type consistency
> >  >> * minimize storage (= current version, since 3.0.0)
> >  >> * maximize utility for large lambda (= proposed change)
> >  >> * keep user interface, and code, simple (e.g., it would be easy
> enough
> >  >> to add a switch that provided user control of int vs double
> return value)
> >  >> * backward compatibility
> >
> >  > Last night, it came to my mind that we should do what we have
> >  > been doing in quite a few places in R, the last couple of years:
> >
> >  > Return integer when possible, and switch to return double when
> >  > integers don't fit.
> >
> >  > We've been doing so even for  1:N  (well, now with additional
> ALTREP wrapper),
> >  > seq(), and even the fundamental  length()  function.
> >
> >  > So I sat down and implemented it .. and it seemed to work
> >  > perfectly:  Returning the same random numbers as now, but
> >  > switching to use double (instead of returning NAs) when the
> >  > values are too large.
> >
> >  > I'll probably commit that to R-devel quite soonish.
> >  > Martin
> >
> > Committed in svn rev 77690; this is really very advantageous, as
> > in some cases / applications or even just limit cases, you'd
> > easily get into overflow sitations.
> >
> > The new R 4.0.0 behavior is IMO  "the best of" being memory
> > efficient (integer storage) in most cases (back compatible to R 3.x.x)
> and
> > returning desired random numbers in large cases (compatible to R <=
> 2.x.x).
> >
> > Martin
>
>
> Wunderbar!  Sehr gut gemacht!  ("Wonderful!  Very well done!") Thanks,
> Spencer
> >
> >  >> On 2020-01-20 12:33 p.m., Martin Maechler wrote:
> >   Benjamin Tyner
> >   on Mon, 20 Jan 2020 08:10:49 -0500 writes:
> >  >>>
> >  >>> > On 1/20/20 4:26 AM, Martin Maechler wrote:
> >  >>> >> Coming late here -- after enjoying a proper weekend ;-) --
> >  >>> >> I have been agreeing (with Spencer, IIUC) on this for a long
> >  >>> >> time (~ 3 yrs, or more?), namely that I've come to see it as
> a
> >  >>> >> "design bug" that  rpois() {and similar} must return return
> typeof() "integer".
> >  >>> >>
> >  >>> >> More strongly, I'm actually pretty convinced they should
> return
> >  >>> >> (integer-valued) double instead of NA_integer_   and for that
> >  >>> >> reason should always return double:
> >  >>> >> Even if we have (hopefully) a native 64bit integer in R,
> >  >>> >> 2^64 is still teeny tiny compared .Machine$double.max
> >  >>> >>
> >  >>> >> (and then maybe we'd have .Machine$longdouble.max  which
> would
> >  >>> >> be considerably larger than double.max unless on Windows,
> where
> >  >>> >> the wise men at Microsoft decided to keep their workload
> simple
> >  >>> >> by defining "long double := double" - as 'long double'
> >  >>> >> unfortunately is not well defined by C standards)
> >  >>> >>
> >  >>> >> Martin
> >  >>> >>
> >  >>> > Martin if you are in favor, then certainly no objection from
> me! ;-)
> >  >>>
> >  >>> > So now what about other discrete distributions e.g. could a
> similar
> >  >>> > enhancement apply here?
> >  >>>
> >  >>>
> >  >>> >> rgeom(10L, 1e-10)
> >  >>> >  [1] NA 1503061294 NA NA
> 1122447583 NA
> >  >>> >  [7] NA NA NA NA
> >  >>> > Warning message:
> >  >>> > In rgeom(10L, 1e-10) : NAs produced
> >  >>>
> >  >>> yes, of course there are several such distributions.
> >  >>>
> >  >>> It's really something that should be discussed (possibly not
> >  >>> here, .. but then I've started it here ...).
> >  >>>
> >  >>> The  NEWS  for R 3.0.0 contain (in NEW FEATURES) :
> >  >>>
> >  >>> * Functions rbinom(), rgeom(), rhyper(), rpois(), rnbinom(),
> >  >>> rsignrank() and rwilcox() now return integer (not double)
> >  >>> vectors.  This halves the storage requirements for large
> >  >>> simulations.
> >  >>>
> >  >>> and what I've been suggesting is to revert this change
> >  >>> (svn rev r60225-6) which was purposefully and diligently done by
> >  >>> a fellow R core member, so indeed must be debatable.
> >  >>>
> >  >>> Martin
> >  >>>
> >  >>> __
> >  >>> R-devel@r-project.org mailing list
> >  >>> https://stat.ethz.ch/mailman/listinfo/r-devel
> >  >>>
> >
> >  >> __
> >  >

Re: [Rd] Rtools and R 4.0.0?

[Avraham Adler]

Absolutely; this is a complicated and frustrating procedure, and we
owe Jeoren and all our gratitude!

Avi

On Tue, Apr 28, 2020 at 3:37 PM Gabriel Becker  wrote:
>
>  Huge thanks to you (Jeroen) and R-core for doing this.
>
> I wasn't involved with this directly but I know it was a pretty seriously
> heavy list so well done all around!
>
> ~G
>
>
>
> On Tue, Apr 28, 2020, 11:04 AM Hervé Pagès  wrote:
>
> > Thanks Jeroen!
> >
> > > On Tue, Apr 7, 2020 at 6:07 PM Kevin Ushey  wrote:
> > >>
> > >> Regardless, I would like to thank R core, CRAN, and Jeroen for all of
> > >> the time that has gone into creating and validating this new
> > >> toolchain. This is arduous work at an especially arduous time, so I'd
> > >> like to voice my appreciation for all the time and energy they have
> > >> spent on making this possible.
> >
> > Absolutely. Thanks to R core, CRAN, Jeroen, and all the other people
> > involved in creating the new Windows toolchain.
> >
> > Cheers,
> > H.
> >
> > >>
> > >> Best,
> > >> Kevin
> > >>
> > >> On Tue, Apr 7, 2020 at 7:47 AM Dirk Eddelbuettel 
> > wrote:
> > >>>
> > >>>
> > >>> There appears to have been some progress on this matter:
> > >>>
> > >>> -Note that @command{g++} 4.9.x (as used for @R{} on Windows up to
> > 3.6.x)
> > >>> +Note that @command{g++} 4.9.x (as used on Windows prior to @R{} 4.0.0)
> > >>>
> > >>> See SVN commit r78169 titled 'anticipate change in Windows toolchain',
> > or the
> > >>> mirrored git commit at
> > >>>
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wch_r-2Dsource_commit_bd674e2b76b2384169424e3d899fbfb5ac174978&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=zMjaTujju0afmK5eIVPZrNajypj8QjuNbSyoAv93ISk&s=oQL_LnqplfOV3qS3_v0vWloGk5Qhr6pWl4Yjzs4Tzzo&e=
> > >>>
> > >>> Dirk
> > >>>
> > >>> --
> > >>>
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__dirk.eddelbuettel.com&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=zMjaTujju0afmK5eIVPZrNajypj8QjuNbSyoAv93ISk&s=nOplDwpoh_urogK65Old_l1Qi-EbVpyC0Mv4LgeLl64&e=
> > | @eddelbuettel | e...@debian.org
> > >>>
> > >>> __
> > >>> R-devel@r-project.org mailing list
> > >>>
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=zMjaTujju0afmK5eIVPZrNajypj8QjuNbSyoAv93ISk&s=vUQZdkVyqq3iT9HukcKqEjg80sI-OZoKuy9DKiufquw&e=
> > >>
> > >> __
> > >> R-devel@r-project.org mailing list
> > >>
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=zMjaTujju0afmK5eIVPZrNajypj8QjuNbSyoAv93ISk&s=vUQZdkVyqq3iT9HukcKqEjg80sI-OZoKuy9DKiufquw&e=
> > >
> > > __
> > > R-devel@r-project.org mailing list
> > >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=zMjaTujju0afmK5eIVPZrNajypj8QjuNbSyoAv93ISk&s=vUQZdkVyqq3iT9HukcKqEjg80sI-OZoKuy9DKiufquw&e=
> > >
> >
> > --
> > Hervé Pagès
> >
> > Program in Computational Biology
> > Division of Public Health Sciences
> > Fred Hutchinson Cancer Research Center
> > 1100 Fairview Ave. N, M1-B514
> > P.O. Box 19024
> > Seattle, WA 98109-1024
> >
> > E-mail: hpa...@fredhutch.org
> > Phone:  (206) 667-5791
> > Fax:(206) 667-1319
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] edit() doubles backslashes when keep.source=TRUE

[Avraham Adler]

I build windows binaries from source. As of now, the only choice is R-revel
unless I want to monkey around more with Jeroens’s PKGBUILD script (which
is On my to-do list).

It’s pretty straightforward, although I’m seeing a lot of issues with
packages which had explicit calls to LOCALSOFT in configure.win as that
doesn’t exist anymore.

The binaries have passed make check, though. Would it help if I built some
and forwarded it somewhere?

Avi

On Fri, May 15, 2020 at 12:48 PM brodie gaslam via R-devel <
r-devel@r-project.org> wrote:

>
> > On Friday, May 15, 2020, 12:13:04 PM EDT, Dirk Eddelbuettel <
> e...@debian.org> wrote:
> > On 15 May 2020 at 15:41, Martin Maechler wrote:
> > | 
> > |
> > |Why does nobody anymore  help R development by working with
> > |"R-devel", or at least then the alpha, beta and the "RC"
> > |(Release Candidate) versions that we release daily for about one
> > |month before the final release?
> > |
> > |Notably a highly staffed enterprise such as Rstudio (viz the bug
> > |report 17800 above), but also others could really help by
> > |starting to use the "next version" of R on a routine basis ...
> > |
> > | 
> >
> > Seconded. Without testing we can never know. R Core does their part.
> >
> > I provided weekly Debian binaries. One each for the two alphas releases,
> for
> > the beta release, for the release candidate.  It is easy to use these,
> for
> > example in a Docker container.
> >
> > It is also easy to use this on a normal machine as they are standard
> (Debian)
> > packages: install, try some tests, uninstall, revert to previous version
> by
> > installing that.
> >
> > Dirk
>
> This is a very reasonably request, and all useRs who benefit from the
> tireless work of R-core should consider doing it.  I have considered
> it, but compiling R from sources on OS X has been my stumbling block.
> At least last time I tried I got stuck at the  Fortran step. It doesn't
>  help I have very limited experience compiling  software of the complexity
> of R.  Really, I've only done it within the warm welcoming confines of the
>  vagrant image Tomas Kalibera set up for `rchk`.
>
> I also use r-devel on docker, but that isn't very practical for
> day-to-day usage, which is what I think we need.
>
> What would it take to generate pre-release binaries for OS X (and
> Windows)?  I
> imagine if such were available the volume of testers would increase
> dramatically (at least, I haven't seen them if they exist).
> Maybe something the R Consortium would consider funding?
>
> Best,
>
> B.
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Improving string concatenation

[Avraham Adler]

On Wed, Jun 17, 2015 at 1:25 AM, Joshua Bradley  wrote:
> …I work in the bioinformatics domain and write R scripts for
> pipelines with calls to various programs that require a lot of parameters
> to be set/varied. Seeing "paste" everywhere detracts from reading the code
> (in my opinion).

In that case, why don't you just write a personal package with that
functionality and automatically load it?

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] GCC update in Rtools?

[Avraham Adler]

Hello.

There was a lot of discussion in March about the difficulties in
having Rtools use a more recent version of GCC than 4.6.3. May we know
if there has been any progress since then, or has dveleopment/testing
been put on hold for the time being?

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Has the changelog for R-devel and R-patched moved?

[Avraham Adler]

Hello.

There used to be changelog of sorts for R-devel [1] and R-release [2].
Neither have been updated since 2015-07-24. Have these moved
elsewhere, or are they no longer being updated?

Thank you,

Avi

[1] 
[2] 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Has the changelog for R-devel and R-patched moved?

[Avraham Adler]

On Tue, Sep 1, 2015 at 10:57 AM, peter dalgaard  wrote:
>
> On 01 Sep 2015, at 16:21 , Avraham Adler  wrote:
>
>> Hello.
>>
>> There used to be changelog of sorts for R-devel [1] and R-release [2].
>> Neither have been updated since 2015-07-24. Have these moved
>> elsewhere, or are they no longer being updated?
>
> Presumably, something that used to happen daily on the hosting machine (or 
> elsewhere) isn't happening any more
>
> If the functionality is easily restored, it will probably be restored soon; 
> if not, probably somewhat later.
>
> Meanwhile, the link to the latest NEWS file (one line above!) appears to be 
> up-to-yesterday's date.
>
> -pd
>
>>
>> Thank you,
>>
>> Avi
>>
>> [1] <http://developer.r-project.org/blosxom.cgi/R-devel/NEWS>
>> [2] <http://developer.r-project.org/blosxom.cgi/R-3-2-branch/NEWS>
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
>

Yes, understood. The reason the changlog is prefereable is that the
NEWS is cumulative, and does not indicate changes between
R-patched_2015-08-26 and R-patched_2015-08-27. Whereas more
information can be gleaned from the changelog entries.


Obviously, this is of very low priority, and if the server refuses to
acknowledge Dr. Murdoch's well-placed steel-toe reminders, getting it
restarted should be deferred as long as is necessary. 8-)

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] R-devel_2015-09-14 throws an error in reg-packages test

[Avraham Adler]

Hello.

I have successfully built R on windows scores of times, and have never
come across this problem. For this build, I am using the current 4.6.3
prerelease version of Rtools, Windows7 64 bit, and
R-devel_2015-09-14.tar.gz

When running make check, I get the following error:


running code in 'reg-packages.R' ...make[3]: *** [reg-packages.Rout] Error 1
make[2]: *** [test-Reg] Error 2
make[1]: *** [test-all-basics] Error 1
make: *** [check] Error 2


Looking at reg-packages.Rout.fail shows:

>
> ## else w/o clause:
> ## could use file.copy(recursive = TRUE)
> system(paste('cp -R', shQuote(pkgSrcPath), shQuote(tempdir(
> pkgPath <- file.path(tempdir(), "Pkgs")
> ## pkgB tests an empty R directory
> dir.create(file.path(pkgPath, "pkgB", "R"), recursive = TRUE,
+showWarnings = FALSE)
> p.lis <- if("Matrix" %in% row.names(installed.packages(.Library)))
+  c("pkgA", "pkgB", "exNSS4") else "exNSS4"
> for(p. in p.lis) {
+ cat("building package", p., "...\n")
+ r <- build.pkg(file.path(pkgPath, p.))
+ cat("installing package", p., "using file", r, "...\n")
+ ## we could install the tar file ... (see build.pkg()'s definition)
+ install.packages(r, lib = "myLib", repos=NULL, type = "source")
+ stopifnot(require(p.,lib = "myLib", character.only=TRUE))
+ detach(pos = match(p., sub("^package:","", search(
+ }
building package pkgA ...
Error: dir.exists(dir) is not TRUE
Execution halted

This error does NOT occur when building R-patched_2015-09-13, so it
must be something in R-devel.

A similar question was asked previously in 2012, once on r-help [1]
and once here [2], but did not receive any answer.

Any and all suggestions would be greatly appreciated.

Thank you,

Avi

[1] 
[2] 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Building manuals are failing now that MikTex 2.9 has removed texi2dvi.exe

[Avraham Adler]

According to the MikTex bug reports [1], MikTex 2.9 has removed
texi2dvi.exe last week (on 2015-09-29) as "it was not compatible
(anymore) with the original shell script texi2dvi (GNU Texinfo)." I
found this out the hard way as I just (unknowingly) updated MikTex
this evening, and then, while building R-devel_2015-10-08 on Windows7
64bit using the Rtools 3.3 toolchain (4.9.3 branch), had it crash
during `make manuals` with:

Output written on fullrefman.pdf (3468 pages, 9511515 bytes).
Transcript written on fullrefman.log.
creating doc/manual/version.texi
texi2dvi --pdf --texinfo="@set UseExternalXrefs " R-FAQ.texi
texi2dvi: not found
make[1]: *** [R-FAQ.pdf] Error 127
make: *** [manuals] Error 2

Outside of trying to dig up an old (and now obsolete) version of the
executable, what can be, or should be, done to build the manuals, or
have we lost that ability?

Thank you,

Avi

[1] http://sourceforge.net/p/miktex/bugs/2400/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Building manuals are failing now that MikTex 2.9 has removed texi2dvi.exe

[Avraham Adler]

On Fri, Oct 9, 2015 at 1:44 AM, Prof Brian Ripley  wrote:
> Hmm, look in MkRules.dist for the setting you failed to make  Obviously
> available in R-patched and R-devel only as we cannot foretell such changes.
>
>[snip]

Yes, I see my error. For efficiency, I had copied in an old pre-filled
Mkrules.local; penny wise and pound foolish.

> Note what the posting guide says about not abusing the word 'crash'.

My apologies for the word "crash." More accurate would have been
"properly halted."

Thank you very much,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Error building Tcl: R-patched_2016-07-05

[Avraham Adler]

I am trying to build R under 64bit Windows7. I am using a fresh
install of Rtools34 and R-patched_2016-07-05. I am getting the
following error:

C:/Rtools/mingw_64/bin/gcc -shared -s -static-libgcc -o tcltk.dll tmp.def init.o
 tcltk.o tcltk_win.o ../../../gnuwin32/dllversion.o -L../../../../Tcl/bin64 -ltc
l85 -ltk85 -LC:/R/RLocalSoft/lib/x64 -LC:/R/RLocalSoft/lib -L../../../../bin/x64
 -lR
C:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w6
4-mingw32/bin/ld.exe: cannot find -ltcl85
C:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w6
4-mingw32/bin/ld.exe: cannot find -ltk85
collect2.exe: error: ld returned 1 exit status
cp: cannot stat 'tcltk.dll': No such file or directory


Looking into R64/Tcl/bin64, I see that the versions provided are
tcl86.dll and tk86.dll, which probably means that line 85 of
R_HOME/src/gnuwin32/fixed/etc/Makeconf needs to be changed from:

TCL_VERSION = 85

to

TCL_VERSION = 86

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Error building Tcl: R-patched_2016-07-05

[Avraham Adler]

OK, that makes sense. Perhaps Rtools34 should then say it should be
used for a version later than 3.3.x? Then again, anyone with the
question will probably find this thread.

Thank you,

Avi

On Thu, Jul 7, 2016 at 6:33 PM, Duncan Murdoch  wrote:
> On 07/07/2016 5:47 PM, Avraham Adler wrote:
>>
>> I am trying to build R under 64bit Windows7. I am using a fresh
>> install of Rtools34 and R-patched_2016-07-05. I am getting the
>> following error:
>>
>> C:/Rtools/mingw_64/bin/gcc -shared -s -static-libgcc -o tcltk.dll tmp.def
>> init.o
>>  tcltk.o tcltk_win.o ../../../gnuwin32/dllversion.o
>> -L../../../../Tcl/bin64 -ltc
>> l85 -ltk85 -LC:/R/RLocalSoft/lib/x64 -LC:/R/RLocalSoft/lib
>> -L../../../../bin/x64
>>  -lR
>>
>> C:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w6
>> 4-mingw32/bin/ld.exe: cannot find -ltcl85
>>
>> C:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w6
>> 4-mingw32/bin/ld.exe: cannot find -ltk85
>> collect2.exe: error: ld returned 1 exit status
>> cp: cannot stat 'tcltk.dll': No such file or directory
>>
>>
>> Looking into R64/Tcl/bin64, I see that the versions provided are
>> tcl86.dll and tk86.dll, which probably means that line 85 of
>> R_HOME/src/gnuwin32/fixed/etc/Makeconf needs to be changed from:
>>
>> TCL_VERSION = 85
>>
>> to
>>
>> TCL_VERSION = 86
>
>
> That is the current setting in R-devel, and has been since revision 70701 on
> June 3.  (It is 85 in the 3.3 branch.)
>
> Duncan Murdoch
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Is it intentional that the Fortran optimization flags changefrom O3 to O2 on Windows?

[Avraham Adler]

When compiling R-devel (2016-08-07) on Windows (64bit) using Rtools
(3.4.0), the C++ optimization flags are manually changed to -O2 from
-O3. This has been the situation for years, and I believe this is to
prevent certain optimizations which may cause downstream problems. In
R_HOME/src/gnuwin32/fixed/etc/Makeconf, this is seen by the -O2
entries in the CXXFLAGS and CXX1XFLAGS variables. The FFLAGS and
FCFLAGS entries in Makeconf still show -O3.

When looking at R_HOME/etc/x64/Makeconf, however, the entries for
FCFLAGS and FFLAGS are -O2. My hunch is that in
R_HOME/src/gnuwin32/fixed/Makefile, line 29 is changing ALL O3 to O2,
and not just those relating to C++:


  ifeq "$(WIN)" "64"
$(SED) -e 's/WIN = 32/WIN = 64/' \
  -e "s/-O3/-O2/" \
  ...


Is this intentional? If not, is there a way to keep O3 optimization
with FORTRAN outside of quickly editing Makeconf before R's
compilation gets to using it?

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Setting Gnu++11 when compiling R-devel on Windows

[Avraham Adler]

Recently, there have been changes to R-devel to make it more
compatible with GCC 6.x, which is great. Unfortunately, Windows still
uses a toolset based on GCC 4.9.3.

When compiling R release or R-patched, one can have GCC called with
-std=gnu++11 by having it in the CXXFLAGS in one's HOME/.R/Makevars as
well as by overwriting CXX1XSTD in
R_HOME/src/gnuwin32/fixed/etc/Makeconf.

When trying the same procedure for R-devel (08-04 and 08-07) I see
that g++ is called without -std=gnu++11. I tried adding the call to
the CXX1YSTD flag as well (although that should be reserved for C++14)
and it did not help.

I can probably force it by adding it to the CXXFLAGS and CXX1XFLAGS in
Makeconf, but that seems to be somewhat overkill and may have
downstream effects of which I am unaware.

Am I correct in my assumption that the new CXXSTD macro as discussed
in the changelogs is automatically setting GCC 4.9.3 to C++98, and if
so, is there a similar way to R-3.3 to have GCC default to gnu++11
outside of forcing it in the general flags? If I am incorrect, I would
appreciate being pointed in the proper direction.

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] optim(…?=, =?utf-8?Q?method=‘L-BFGS-B’) stops with an error message while violating the lower bound

[Avraham Adler]

I believe the code can be found here:
http://users.iems.northwestern.edu/~nocedal/lbfgsb.html. Specifically,
lbfgsb.f in version 3.0 starts:

This is a modified version of L-BFGS-B. Minor changes in the updated
c code appear preceded by a line comment as follows
c
c c-jlm-jn
c
c Major changes are described in the accompanying paper:
c
c Jorge Nocedal and Jose Luis Morales, Remark on "Algorithm 778:
c L-BFGS-B: Fortran Subroutines for Large-Scale Bound Constrained
c Optimization"  (2011). To appear in  ACM Transactions on
c Mathematical Software,
c
c The paper describes an improvement and a correction to Algorithm 778.
c It is shown that the performance of the algorithm can be improved
c significantly by making a relatively simple modication to the subspace
c minimization phase. The correction concerns an error caused by the use
c of routine dpmeps to estimate machine precision.


It is released under the New 3-clause BSD license, so porting it to C
for inclusion into R should be OK as long as the i's are dotted and
t's crossed.


Avi

On Mon, Oct 10, 2016 at 5:54 AM, Martin Maechler
 wrote:
>> Spencer Graves 
>> on Sat, 8 Oct 2016 18:03:43 -0500 writes:
>
> [.]
>
> >  2.  It would be interesting to know if the
> > current algorithm behind optim and optimx with
> > method='L-BFGS-B' incorporates Morales and Nocedal (2011)
> > 'Remark on “Algorithm 778: L-BFGS-B: Fortran Subroutines
> > for Large-Scale Bound Constrained Optimization”'.  I
> > created this vignette and started this threat hoping that
> > someone on the R Core team might decide it's worth
> > checking things like that.
>
> well I hope you mean "thread" rather "threat"  ;-)
>
> I've now looked at the reference above, which is indeed quite
> interesting.
> doi 10.1145/2049662.2049669
> --> http://dl.acm.org/citation.cfm?doid=2049662.2049669
> A "free" (pre-publication I assume) version of the manuscript is
>   http://www.eecs.northwestern.edu/~morales/PSfiles/acm-remark.pdf
>
> The authors, Morales and Nocedal, the 2nd one being one of the
> original L-BFGS-B(1997) paper, make two remarks, the 2nd one
> about the "machine epsilon" used, and I can assure you that R's
> optim() version never suffered from that; we've always been
> using a C translation of the fortran code, and then used DBL_EPSILON.
> R's (main) source file for that is in .../src/appl/lbfgsb.c, e.g., here
> https://svn.r-project.org/R/trunk/src/appl/lbfgsb.c
>
> OTOH, their remark 1 is very relevant and promising faster /
> more reliable convergence.
> I'd be "happy" if optim() could gain a new option, say, "L-BFGS-B-2011"
> which would incorporate what they call "modified L-BFGS-B".
>
> However, I did not find published code to go together with their
> remark.
> Ideally, some of you interested in this, would provide a patch
> against the above  lbfgsb.c  file
>
> Martin Maechler,
> ETH Zurich
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] unlicense

[Avraham Adler]

A number of years ago I asked here for the ISC to be added and was told you
have to ask CRAN, not Rd.

Good luck,

Avi

On Fri, Jan 13, 2017 at 3:22 PM Charles Geyer  wrote:

> I would like the unlicense (http://unlicense.org/) added to R
>
> licenses.  Does anyone else think that worthwhile?
>
>
>
> --
>
> Charles Geyer
>
> Professor, School of Statistics
>
> Resident Fellow, Minnesota Center for Philosophy of Science
>
> University of Minnesota
>
> char...@stat.umn.edu
>
>
>
> __
>
> R-devel@r-project.org mailing list
>
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> --
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Unexpected EOF in R-patched_2017-01-30

[Avraham Adler]

Hello.

When trying to unpack today's version of R-patched, I get the following error:

C:\R>tar -xf R-patched_2017-01-30.tar.gz

gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

I got the same error for R-patched_2017-01-30.tar.gz but not for R-3.3.2.tar.gz.

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Unexpected EOF in R-patched_2017-01-30

[Avraham Adler]

On Tue, Jan 31, 2017 at 3:30 PM, peter dalgaard  wrote:
>
>> On 31 Jan 2017, at 18:56 , Avraham Adler  wrote:
>>
>> Hello.
>>
>> When trying to unpack today's version of R-patched,
>
> From which source? The files from cran.r-project.org seems OK, both those in 
> src/base-prerelease and those from ETHZ. Also, is it not "tar -xfz" when 
> reading a compressed file?
>
> -pd

>From <https://stat.ethz.ch/R/daily/>

Also, while passing z is not in the instructions given in Installation
and Administration [1], I tried passing -xzf and it did not work. I
believe f has to be last if the file name follows immediately.

[1]  
<https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Getting-the-source-files>

Thanks,

Avi

>> I get the following error:
>>
>> C:\R>tar -xf R-patched_2017-01-30.tar.gz
>>
>> gzip: stdin: unexpected end of file
>> tar: Unexpected EOF in archive
>> tar: Unexpected EOF in archive
>> tar: Error is not recoverable: exiting now
>>
>> I got the same error for R-patched_2017-01-30.tar.gz but not for 
>> R-3.3.2.tar.gz.
>>
>> Thank you,
>>
>> Avi
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Unexpected EOF in R-patched_2017-01-30

[Avraham Adler]

It worked for me at home but not at work. I will email my work the copy I
have at home. It could have been something with our connectivity or
firewall. My home copy of 1-31 is also 388b607afe732c92442dbb49845fe377.
I'll check the work ones tomorrow, or should I say later today.

Thank you,

Avi

On Wed, Feb 1, 2017 at 3:14 AM, Martin Maechler 
wrote:

> >>>>> Avraham Adler 
> >>>>> on Tue, 31 Jan 2017 16:07:20 -0500 writes:
>
> > On Tue, Jan 31, 2017 at 3:30 PM, peter dalgaard 
> wrote:
>     >>
> >>> On 31 Jan 2017, at 18:56 , Avraham Adler 
> wrote:
> >>>
> >>> Hello.
> >>>
> >>> When trying to unpack today's version of R-patched,
> >>
> >> From which source? The files from cran.r-project.org seems OK,
> both those in src/base-prerelease and those from ETHZ.
>
> >> Also, is it not "tar -xfz" when reading a compressed file?
>
> Recent (for several years) versions of tar (on Linux at least)
> do not need the compression extension anymore: They guess it
> correctly from the file.
>
> >>
> >> -pd
>
> >> From <https://stat.ethz.ch/R/daily/>
>
> The last two of the daily R-patched*.tar.gz
> unpack flawlessly for me as well.
>
> Could it be that your Windows(?) version of tar (or the file
> system or ???) is the problem?
>
> Or the file was corrupted during download?
>
> Here are the md5sum s  from the server itself for the last three snapshots:
>
> 388b607afe732c92442dbb49845fe377  /ftp/R/R-patched_2017-01-31.tar.gz
> 7daea59067454311818df1c75971a485  /ftp/R/R-patched_2017-01-30.tar.gz
> 9ddad833a455973631920c70b6da5d6e  /ftp/R/R-patched_2017-01-29.tar.gz
>
>
>
> > Also, while passing z is not in the instructions given in
> Installation
> > and Administration [1], I tried passing -xzf and it did not work. I
> > believe f has to be last if the file name follows immediately.
>
> > [1]  <https://cran.r-project.org/doc/manuals/r-release/R-admin.
> html#Getting-the-source-files>
>
> > Thanks,
>
> > Avi
>
> >>> I get the following error:
> >>>
> >>> C:\R>tar -xf R-patched_2017-01-30.tar.gz
> >>>
> >>> gzip: stdin: unexpected end of file
> >>> tar: Unexpected EOF in archive
> >>> tar: Unexpected EOF in archive
> >>> tar: Error is not recoverable: exiting now
> >>>
> >>> I got the same error for R-patched_2017-01-30.tar.gz but not for
> R-3.3.2.tar.gz.
> >>>
> >>> Thank you,
> >>>
> >>> Avi
> >>>
> >>> __
> >>> R-devel@r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
> >> --
> >> Peter Dalgaard, Professor,
> >> Center for Statistics, Copenhagen Business School
> >> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> >> Phone: (+45)38153501
> >> Office: A 4.23
> >> Email: pd@cbs.dk  Priv: pda...@gmail.com
> >>
>
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Unexpected EOF in R-patched_2017-01-30

[Avraham Adler]

On Wed, Feb 1, 2017 at 3:14 AM, Martin Maechler 
wrote:

> Or the file was corrupted during download?
>
> Here are the md5sum s  from the server itself for the last three snapshots:
>
> 388b607afe732c92442dbb49845fe377  /ftp/R/R-patched_2017-01-31.tar.gz
> 7daea59067454311818df1c75971a485  /ftp/R/R-patched_2017-01-30.tar.gz
> 9ddad833a455973631920c70b6da5d6e  /ftp/R/R-patched_2017-01-29.tar.gz
>

That is probably the case as the 01-30 version gives me an MD5 of
49fcb4ad0874b136a4499b8d3d39cc03.

Thank you,

Avi

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Registration of native routines

[Avraham Adler]

On Tue, Feb 14, 2017 at 11:25 AM, Prof Brian Ripley
 wrote:
> Registration of 'native routines' (entry points in compiled code loaded into
> R) has been available for over 14 years, but few packages make use of it
> (less than 10% of those on CRAN with compiled code).
>
> Registration has similar benefits to name spaces in R code:
>
> - it ensures that the routines used by .C, .Call etc are those in your
> package (without needing a PACKAGE argument).
>
> - it avoids polluting the search space for native routines with those from
> your package.
>
> - it checks the number of arguments passed to .Call/.External, and the
> number and optionally the type for .C/.Fortran.
>
> - it finds native routines faster, especially if 10s of name spaces are
> loaded.
>
> Kurt Hornik and I have written a tool to make adding registration much
> easier.  From NEWS in R-devel
>
> • Package tools has a new function
>   package_native_routine_registration_skeleton() to assist adding
>   native-routine registration to a package.  See its help and §5.4.1
>   of ‘Writing R Extensions’ for how to use it.  (At the time it was
>   added it successfully automated adding registration to over 90%
>   of CRAN packages which lacked it.  Many of the failures were
>   newly-detected bugs in the packages, e.g. 50 packages called
>   entry points with varying numbers of arguments and 65 packages
>   called entry points not in the package.)

Hello, Dr., Ripley.

This is fantastic. Is there a way to install this functionality into
an existing 3.3.2 installation, or is it exclusive to
R-deve;/R-3.4-to-be?

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] R-devel 2017-02-15: repeated line in /src/gnuwin32/MkRules.dist

[Avraham Adler]

Hello.

Unless I am mistakedn, it seems that lines 49 and 72 are copies of
each other and only one should be necessary to set the whether or not
Windows is 32- or 64-bit. If it were my guess, I'd say lines 48 & 49
are redundant.

Thanks,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Wish List: Extensions to the derivatives table

[Avraham Adler]

Hi.

Unless I'm misremembering, log, exp, sin, cos, and tan are all handled in
deriv3. The functions listed are  specially coded slightly more accurate
versions but can be substituted with native ones for which deriv/deriv3
will work automatically. I believe that if you  write your functions using
log(a + 1) instead of log1p(a) or log(x) / log(2) instead of log2(x) deriv3
will work fine.

Thanks,

Avi

On Fri, Feb 17, 2017 at 2:02 PM Jerry Lewis  wrote:

> The derivative table resides in the function D.  In S+ that table is
> extensible because it is written in the S language.  R is faster but less
> flexible, since that table is programmed in C.  It would be useful if R
> provided a mechanism for extending the derivative table, or barring that,
> provided a broader table.  Currently unsupported mathematical functions of
> one argument include expm1, log1p, log2, log10, cospi, sinpi, and tanpi.
>
> While manual differentiation of these proposed additions is
> straight-forward, their absence complicates what otherwise could be much
> simpler, such as using deriv() or deriv3() to generate functions, for
> example to use as an nls model.
>
> Thanks,
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Registration of native routines

[Avraham Adler]

I believe Dr. Ripley said that the UseDynLib doesn't address the
middle two points of the list.

I experimented with the Delaporte package timing calls using both
methods, and they were within 1~2 microseconds of each other using
microbenchmark. Since the more complicated way is now expressly
preferred, it isn't a big deal for me personally to change so the next
version of Delaporte will use CallMethod/useDynLib(foo,
.registration=TRUE) etc. I can see how in more complicated packages
that could become more burdensome.

Thanks,

Avi

On Wed, Feb 22, 2017 at 9:34 AM, Jeroen Ooms  wrote:
> On Tue, Feb 14, 2017 at 5:25 PM, Prof Brian Ripley
>  wrote:
>>
>> Registration has similar benefits to name spaces in R code:
>>
>> - it ensures that the routines used by .C, .Call etc are those in your 
>> package (without needing a PACKAGE argument).
>> - it avoids polluting the search space for native routines with those from 
>> your package.
>> - it checks the number of arguments passed to .Call/.External, and the 
>> number and optionally the type for .C/.Fortran.
>> - it finds native routines faster, especially if 10s of name spaces are 
>> loaded.
>
> Do these benefits also hold for packages that currently use useDynLib
> exclusively in combination symbol names? E.g for the example from WRE:
>
>useDynLib(foo, myRoutine, myOtherRoutine)
>
> Which is invoked via:
>
>   .Call(myRoutine, x, y)
>
> What ambiguity or pollution is introduced by foo:::myRoutine? Should
> manually registering 'myRoutine' in C still be mandatory in this case?
>
> It was nice having the NAMESPACE as the central declaration of
> callable C routines. The "R_registerRoutines" system will require
> maintaining additional C code which re-declares all callable C
> functions from other compilation units. This introduces additional
> complexity for package authors and might become a source of bugs when
> we forget to update the registrations when C functions have changed.
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R 3.4.0

[Avraham Adler]

Perhaps the darkness is so stupid it forgot to update the year?

Avi
On Fri, Mar 17, 2017 at 12:24 PM  wrote:

> Your dates are for 2016 :-) in your email and developer.r-project.com
>
> Best,
>
> luke
>
> On Fri, 17 Mar 2017, Peter Dalgaard wrote:
>
> > R 3.4.0 "You Stupid Darkness" is now scheduled for April 21
> >
> > The detailed schedule can be found on developer.r-project.org
> >
> > For the Core Team
> >
> > Peter D.
> >
> >
>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa  Phone: 319-335-3386
> Department of Statistics andFax:   319-335-3017
> Actuarial Science
> 241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
> Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] TEXINFO error when building R-3.4.0

[Avraham Adler]

Hello.

I am trying to build 64 bit R-3.4.0 (specifically
R-patched_2017-04-23) on Windows 64 and have run into an issue with
building the html files. I am using Rtools34 version 1962, the texinfo
zip from  unzipped into
C:\R\texinfo, and Strawberry perl 5.24.1.1-64bit. In my MkRules.local
I have the following, which worked for building R in 3.3.x:

# set this to YES to build static HTML help
BUILD_HTML = YES

# unset this if you are *not* using MiKTeX
MIKTEX = TRUE
# Recent MiKTEX does not provide texi2dvi and needs something like
TEXI2DVI = TEXINDEX=C:/Rtools/bin/texindex.exe texify

# for texinfo >= 5.1. If the texinfo files are installed at /packages/texinfo,
TEXI2ANY = C:/Strawberry/perl/bin/perl.exe -I/C:/R/texinfo C:/R/texinfo/texi2any
# if you do not have texinfo (default),
# TEXI2ANY = missing

Nevertheless, while building R-3.4 I get the following error:


-- Making HTML documentation --
creating doc/manual/version.texi
creating doc/manual/R-FAQ.html
Can't locate Texinfo/Convert/Texinfo.pm in @INC (you may need to install the Tex
info::Convert::Texinfo module) (@INC contains: \usr\local\share\texinfo\lib\Text
-Unidecode\lib \usr\local\share\texinfo\lib\Unicode-EastAsianWidth\lib \usr\loca
l\share\texinfo\lib\libintl-perl\lib \usr\local\share\texinfo /C:/R/texinfo C:/S
trawberry/perl/site/lib C:/Strawberry/perl/vendor/lib C:/Strawberry/perl/lib .)
at C:/R/texinfo/texi2any line 100.
BEGIN failed--compilation aborted at C:/R/texinfo/texi2any line 100.
sed: can't read R-FAQ.html.tmp: No such file or directory
make[3]: *** [R-FAQ.html] Error 2
make[2]: *** [htmldocs] Error 2
make[1]: *** [all] Error 2
make: *** [distribution] Error 2

If it helps, Convert seems to be found in C:\R\texinfo\Texinfo\Convert
which is not in the list of @INC. Any and all help would be
appreciated.

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] TEXINFO error when building R-3.4.0

[Avraham Adler]

The fault was mine.

-I/C:/R/texinfo should have been -IC:/R/texinfo

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Philosophy behind converting Fortran to C for use in R

[Avraham Adler]

Hello.

This is not a question about a bug or even best practices; rather I'm
trying to understand the philosophy or theory as to why certain
portions of the R codebase are written as they are. If this question
is better posed elsewhere, please point me in the proper direction.

In the thread about the issues with the Tukey line, Martin said [1]:

> when this topic came up last (for me) in Dec. 2014, I did spend about 2 days 
> work (or more?)
> to get the FORTRAN code from the 1981 - book (which is abbreviated the "ABC 
> of EDA")
> from a somewhat useful OCR scan into compilable Fortran code and then f2c'ed,
> wrote an R interface function found problems…

I have seen this in the R source code and elsewhere, that native
Fortran is converted to C via f2c and then run as C within R. This is
notwithstanding R's ability to use Fortran, either directly through
.Fortran() [2] or via .Call() using simple helper C-wrappers [3].

I'm curious as to the reason. Is it because much of the code was
written before Fortran 90 compilers were freely available? Does it
help with maintenance or make debugging easier? Is it faster or more
likely to compile cleanly?

Thank you,

Avi

[1] https://stat.ethz.ch/pipermail/r-devel/2017-May/074363.html
[2] Such as kmeans does for the Hartigan-Wong method in the stats package
[2] Such as the mvtnorm package does

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Solaris SPARC testbed

[Avraham Adler]

I've looked at some package testing results and I'm not seeing Solaris
SPARC. Has that test-bed been deprecated for package testing?

Thanks,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Duncan's retirement: who's taking over Rtools?

[Avraham Adler]

On Thu, Sep 28, 2017 at 11:27 AM, Joris Meys  wrote:
> Dear dev team,
>
> I was sorry to see the announcement of Duncan about his retirement from
> maintaining the R Windows build and Rtools. Duncan, thank you incredibly
> much for your 15 years of devotion and your impressive contribution to the
> R community as a whole.
>
> Thinking about the future, I wondered whether there were plans for the
> succession of Duncan. Is it the intention to continue providing Rtools and
> a Windows build, or are these tasks left open for anyone (possibly
> Microsoft itself) to take them over? And if so, how will the decision be
> made on that?
>

Always willing to volunteer someone else, my first thoughts would be
to contact Jeroen Ooms and Kevin Ushey. Both, especially Jeroen, were
integral to the successful deployemnt of the 4.9.3 version of Rtools.

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Duncan's retirement: who's taking over Rtools?

[Avraham Adler]

On Thu, Sep 28, 2017 at 11:47 AM, David Smith via R-devel
 wrote:
> The Microsoft R team is willing and able to produce builds for R on Windows 
> going forward. As Duncan noted, we've been doing this already for some time 
> for MRAN. I'd love to hear thoughts from this community on what that might 
> mean, and Duncan I'll also reach out to you directly off-list.

Hi, David.

If the Microsoft R team takes over that responsibility, will they also
take over the testing and maintenance of the scripts for building R
from source on Windows as Duncan has?

Thanks,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Debate: Shall some of Microsoft R Open Code be ported to mainstream R?

[Avraham Adler]

[Sent offlist accidentally]

What concerns me first and foremost is that the licensure would have
to be ironclad (including for commercial use like vanilla R now) as
well as ensuring that R remains completely FLOSS. Anything “added” to
R has to be a no-strings-attached gift to R.

Also, I would think that it would have to play nice with existing
workflows (like OpenBLAS instead of MKL) unless there is such a
benefit that it is worth breaking compatibility.

Avi

On Sun, Oct 29, 2017 at 6:01 PM, Kenny Bell  wrote:
> User here: incorporating Intel's MKL, as MRO does, would be a very welcome
> addition.
>
> I was an MRO user before and it improved my experience with medium data
> immensely.
>
> They did, however, leave behind bugs here and there, especially related to
> development with Rcpp, so I switched back to vanilla R.
>
> On Mon, Oct 30, 2017, 9:42 AM Juan Telleria  wrote:
>
>> Dear R Developers,
>>
>> First of all, I would like to thank you Jeroen Ooms for taking the binary
>> Window Builds from Duncan. I firmly believe that the R Community will
>> benefit a lot from his work.
>>
>> However, the debate I would like to open is about if some of Microsoft R
>> Open Code shall be ported from R Open to Mainstream R.
>>
>> There are some beneficts in R Open such as multithreaded performance:
>> https://mran.microsoft.com/documents/rro/multithread/
>>
>> Maybe, the R Consortium, and in particular, Microsoft R Team, could
>> collaborate, if appropriate, in such duty.
>>
>> Thank you,
>> Juan Telleria
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Debate: Shall some of Microsoft R Open Code be ported to mainstream R?

[Avraham Adler]

On Mon, Oct 30, 2017 at 12:45 PM, Cohn, Robert S
 wrote:
> I think the thing that is missing is a simple way for end users on windows to 
> replace blas/lapack libraries with MKL-a package that you install that puts 
> the libraries in the right place.
>
> Microsoft provides something for their distro, but we don't have the 
> equivalent if you get R from cran.
>

Dr. Ei-ji Nakama has some compiled GotoBLAS Rblas DLLs on his site
[1], but they are rather old now. I've toyed with the idea of doing
something similar for OpenBLAS—which requires tricking R into thinking
it's ATLAS [2]—and hosting at least a Sandy Bridge and a dynamic
version. I'm not sure how the licenses and copyrights would interact
though. Also, that only works for the BLAS. Using an optimized LAPACK
is even more difficult. On Unix-likes, the manual does not recommend
it but it's allowed as an option in the configure [3]. For OpenBLAS
(and ATLAS for that matter), even though one can build an optimized
LAPACK, there is no way to access the build of the LAPACK on Windows
through Mkrules.local, and R_HOME/src/modules/lapack/README makes it
seem to be rather difficult, if not impossible. I believe the MKL
includes a specialized LAPACK; I do not know if this is compiled into
M$ R, though.

Thanks,

Avi

[1] https://prs.ism.ac.jp/~nakama/SurviveGotoBLAS2/binary/windows/
[2] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/
[3] https://cran.r-project.org/doc/manuals/r-release/R-admin.html#LAPACK

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

[Avraham Adler]

On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell  wrote:
> It seems like many of the multi-threaded BLASes have some sort of
> fundamental problem preventing use in the way Juan suggests:
>
>  - Dirk's vignette states that ATLAS "fixes the number of cores used at
> compile-time and cannot vary this setting at run-time", so any
> user-friendly implementation for R would have to compile ATLAS for 1-16
> threads to allow the user to switch at run-time. This might dramatically
> affect install times.
>
>  - MKL seems like it's been outright rejected in the past based on not
> being "free-enough".
>
>  - OpenBLAS causes crashes.
>
> Has anyone tried ExBLAS for use with R?
>
> On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder <
> peter.langfel...@gmail.com> wrote:
>
>> I would be very cautious about OpenBLAS in particular...  from time to
>> time I get complains from users that compiled code calculations in my
>> WGCNA package crash or produce wrong answers with large data, and they
>> all come from OpenBLAS users. I am yet to reproduce any of their
>> crashes when using MKL and ATLAS BLAS implementations.
>>
>> Just my 2 cents...
>>
>> Peter

I've been building R on Windows 64 bit with OpenBLAS for years and my
builds pass check-devel. For a while in the past it failed one check
as the tolerance was 5e-5 and with my build of OpenBLAS the error was
5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall
correctly. I provide descriptions here [1], but I haven't gone so far
as to post compiled Rblas.dlls just yet. My personal build sets 4
threads when compiling OpenBLAS itself as I'm currently on a quad-core
SandyBridge. In tests I ran a few years ago, both single and multi
threaded BLAS compile and then can be compiled into R with no issues
(on my platforms, at least). Most matrix operations performed better
with multi-threaded except for R's eigenvalue decomposition, to the
nest of my recollection.

Avi

[1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Debugging packages with compiled C code on Windows

[Avraham Adler]

Hello, Sue.

1. I work exclusively on Windows have packages with compiled C, C++,
and Fortran (2003+)  and I use RStudio to debug and work with them
using Rtools40, so I guess it's possible. Have I misunderstood and are
you asking about debugging assembler?
2. If you're using Jeroen's scripts, have you tried uncommenting and
adding that to EOPTS in MkRules.local.in? Note that
./src/gnuwin32/fixed/Makefile has a nasty habit of overriding various
optimizations that affect packages.
3. I don't think so
4. The default is that EOPTS is commented out. I talk about it a nit
more at length here [1]. Perhaps that would be of use?

[1] 


Good Luck,

Avi


On Mon, Jun 1, 2020 at 9:36 PM Sue McDonald  wrote:
>
> I have several related questions.
>
> 1.  Is it possible to use a GUI: Rstudio/Eclipse/Visual-studio to debug
> compiled code on Windows? Things that work on Eclipse for Windows do not
> work on Eclipse for Windows.
> 2.  R CMD INSTALL seems to override default attempts to provide
> CFLAGS="-DDEBUG -g3 -O0"
> 3.  Is it necessary to compile R with debug turned on?   One of the FAQs
> mentioned to compile R with make DEBUG=T.
> 4.  Using Rtools 4.0 and Jeroen's scripts for building R works great (many
> thanks). But does not seem to have an impact on optimization, other than
> including -gwarf-2.  It adds -DNDEBUG flag.  Is that sufficient for
> debugging compiled code in a package?  Obviously, I just need to debug
> package code, so does it matter?
>
> I am happy to write-up a FAQ.
>
> Thanks, SM
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [External] use of the tcltk package crashes R 4.0.1 for Windows

[Avraham Adler]

Jeroen, how "reactive" are the rtools40 scripts. Will they pull the latest
version committed by Dr. Tierney or is there something which must be done
manually prior to we end-users rebuilding from source?

Thank you,

Avi

On Sun, Jun 7, 2020 at 11:01 PM peter dalgaard  wrote:

> Ah, I see it now:
>
> The remapping of free() to Rm_free() and calloc() to Rm_calloc() happens
> in memory.c, but not in tcltk.c; the macro Calloc in R_ext/RS.h maps to a
> call to R_chk_alloc which is defined in memory.h; RS.h is included in
> tcltk.c, so tcltk.c winds up calling Rm_calloc() via Calloc(), but then the
> NON-remapped free(), and the walls come tumbling down.
>
> If the  "#if defined(Win32)" block had been inside RS.h, the problem
> wouldn't arise.
>
> -pd
>
> > On 8 Jun 2020, at 00:03 , luke-tier...@uiowa.edu wrote:
> >
> > I've committed the change to use Free instead of free in tcltk.c and
> > sys-std.c (r78652 for R-devel, r78653 for R-patched).
> >
> > We might consider either moving Calloc/Free out of the Windows
> > remapping or moving the remapping into header files so everything
> > seeing our header files uses our calloc/free. Either would be less
> > brittle that the current status.
> >
> > Best,
> >
> > luke
> >
> > On Sun, 7 Jun 2020, peter dalgaard wrote:
> >
> >>
> >>
> >>> On 7 Jun 2020, at 18:59 , Jeroen Ooms  wrote:
> >>>
> >>> On Sun, Jun 7, 2020 at 5:53 PM  wrote:
> 
>  On Sun, 7 Jun 2020, peter dalgaard wrote:
> 
> > So this wasn't tested for a month?
> >
> > Anyways, Free() is just free() with a check that we're not freeing a
> null pointer, followed by setting the pointer to NULL. At that point of
> tcltk.c, we have
> >
> > for (objc = i = 0; i < length(avec); i++){
> >  const char *s;
> >  char *tmp;
> >  if (!isNull(nm) && strlen(s = translateChar(STRING_ELT(nm,
> i{
> >  //  tmp = calloc(strlen(s)+2, sizeof(char));
> >  tmp = Calloc(strlen(s)+2, char);
> >  *tmp = '-';
> >  strcpy(tmp+1, s);
> >  objv[objc++] = Tcl_NewStringObj(tmp, -1);
> >  free(tmp);
> >  }
> >  if (!isNull(t = VECTOR_ELT(avec, i)))
> >  objv[objc++] = (Tcl_Obj *) R_ExternalPtrAddr(t);
> >  }
> >
> > and I can't see how tmp can be NULL at the free(), nor can I see it
> mattering if it is not set to NULL (notice that it goes out of scope with
> the for loop).
> 
>  Right. And the calloc->Calloc change doesn't look like an issue either
>  -- just checking for a NULL.
> 
>  If the crash is happening in free() then that most likely means
>  corrupted malloc data structures. Unfortunately that could be
>  happening anywhere.
> >>>
> >>> Writing R extensions, section 6.1.2 says: "Do not assume that memory
> >>> allocated by Calloc/Realloc comes from the same pool as used by
> >>> malloc: in particular do not use free or strdup with it."
> >>>
> >>> I think the reason is that R uses dlmalloc for Calloc on Windows:
> >>>
> https://github.com/wch/r-source/blob/c634fec5214e73747b44d7c0e6f047fefe44667d/src/main/memory.c#L94-L103
> >>
> >> But that section #defines calloc and free to Rm_... counterparts in
> lockstep? (I assume that is where dlmalloc comes in?)
> >>
> >> Anyways, does it actually work to change free() to Free()? If so, then
> all this post mortem analysis is rather a moot point.
> >>
> >> -pd
> >>
> >>
> >
> > --
> > Luke Tierney
> > Ralph E. Wareham Professor of Mathematical Sciences
> > University of Iowa  Phone: 319-335-3386
> > Department of Statistics andFax:   319-335-3017
> >   Actuarial Science
> > 241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
> > Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] From .Fortran to .Call?

[Avraham Adler]

Hello, Ivan.

I used .Call instead of .Fortran in the Delaporte package [1]. What
helped me out a lot was Drew Schmidt's Romp examples and descriptions
[2]. If you are more comfortable with the older Fortran interface,
Tomasz Kalinowski has a package which uses Fortran 2018 more
efficiently [3]. I haven't tried this last in practice, however.

Hope that helps,

Avi

[1] https://CRAN.R-project.org/package=Delaporte
[2] https://github.com/wrathematics/Romp
[3] https://github.com/t-kalinowski/RFI

Tomasz Kalinowski



On Tue, Dec 22, 2020 at 7:24 PM Balasubramanian Narasimhan
 wrote:
>
> To deal with such Fortran issues in several packages I deal with, I
> wrote the SUtools package (https://github.com/bnaras/SUtools) that you
> can try.  The current version generates the registration assuming
> implicit Fortran naming conventions though. (I've been meaning to
> upgrade it to use the gfortran -fc-prototypes-external flag which should
> be easy; I might just finish that during these holidays.)
>
> There's a vignette as well:
> https://bnaras.github.io/SUtools/articles/SUtools.html
>
> -Naras
>
>
> On 12/19/20 9:53 AM, Ivan Krylov wrote:
> > On Sat, 19 Dec 2020 17:04:59 +
> > "Koenker, Roger W"  wrote:
> >
> >> There are comments in various places, including R-extensions §5.4
> >> suggesting that .Fortran is (nearly) deprecated and hinting that use
> >> of .Call is more efficient and now preferred for packages.
> > My understanding of §5.4 and 5.5 is that explicit routine registration
> > is what's important for efficiency, and your package already does that
> > (i.e. calls R_registerRoutines()). The only two things left to add
> > would be types (REALSXP/INTSXP/...) and styles (R_ARG_IN,
> > R_ARG_OUT/...) of the arguments of each subroutine.
> >
> > Switching to .Call makes sense if you want to change the interface of
> > your native subroutines to accept arbitrary heavily structured SEXPs
> > (and switching to .External could be useful if you wanted to play with
> > evaluation of the arguments).
> >
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] From .Fortran to .Call?

[Avraham Adler]

Hi.

I haven't tested the speed of the old .Fortran interface, but in this
post [1] I describe how to build a simple interface (there are two
small packages on github that correspond to the code) and in this one
[2] I compare the speed of the different languages, but all using
.Call.

Hope that helps,

Avi

[1] 
https://www.avrahamadler.com/2018/12/09/the-need-for-speed-part-1-building-an-r-package-with-fortran/
[2] 
https://www.avrahamadler.com/2018/12/23/the-need-for-speed-part-2-c-vs-fortran-vs-c/

On Wed, Dec 23, 2020 at 11:34 AM Balasubramanian Narasimhan
 wrote:
>
> I think it should be pretty easy to fix up SUtools to use the .Call
> instead of .Fortran following along the lines of
>
> https://github.com/wrathematics/Romp
>
> I too deal with a lot of f77 and so I will most likely finish it before
> the new year, if not earlier. (Would welcome testers besides myself.)
>
> Incidentally, any idea of what the performance hit is, quantitatively? I
> confess I never paid attention to it myself as most Fortran code I use
> seems pretty fast, i.e. glmnet.
>
> -Naras
>
>
> On 12/23/20 3:57 AM, Koenker, Roger W wrote:
> > Thanks to all and best wishes for a better 2021.
> >
> > Unfortunately I remain somewhat confused:
> >
> >   o  Bill reveals an elegant way to get from my rudimentary  
> > registration setup to
> >   one that would explicitly type the C interface functions,
> >
> >   o Ivan seems to suggest that there would be no performance gain from 
> > doing this.
> >
> >   o  Naras’s pcLasso package does use the explicit C typing, but then 
> > uses .Fortran
> >   not .Call.
> >
> >   o  Avi uses .Call and cites the Romp package 
> > https://github.com/wrathematics/Romp
> >   where it is asserted that "there is a (nearly) deprecated interface 
> > .Fortran() which you
> >   should not use due to its large performance overhead.”
> >
> > As the proverbial naive R (ab)user I’m left wondering:
> >
> >   o  if I updated my quantreg_init.c file in accordance with Bill’s 
> > suggestion could I
> >   then simply change my .Fortran calls to .Call?
> >
> >   o  and if so, do I need to include ALL the fortran subroutines in my 
> > src directory
> >   or only the ones called from R?
> >
> >   o  and in either case could I really expect to see a significant 
> > performance gain?
> >
> > Finally, perhaps I should stipulate that my fortran is strictly f77, so no 
> > modern features
> > are in play, indeed most of the code is originally written in ratfor, Brian 
> > Kernighan’s
> > dialect from ancient times at Bell Labs.
> >
> > Again,  thanks to all for any advice,
> > Roger
> >
> >
> >> On Dec 23, 2020, at 1:11 AM, Avraham Adler  wrote:
> >>
> >> Hello, Ivan.
> >>
> >> I used .Call instead of .Fortran in the Delaporte package [1]. What
> >> helped me out a lot was Drew Schmidt's Romp examples and descriptions
> >> [2]. If you are more comfortable with the older Fortran interface,
> >> Tomasz Kalinowski has a package which uses Fortran 2018 more
> >> efficiently [3]. I haven't tried this last in practice, however.
> >>
> >> Hope that helps,
> >>
> >> Avi
> >>
> >> [1] 
> >> https://urldefense.com/v3/__https://CRAN.R-project.org/package=Delaporte__;!!DZ3fjg!s1-ihrZ9DPUtXpxdIpJPA1VedpZFt12Ahmn4CycOmile_uSahFZnJPn_5KPITBN5NK8$
> >> [2] 
> >> https://urldefense.com/v3/__https://github.com/wrathematics/Romp__;!!DZ3fjg!s1-ihrZ9DPUtXpxdIpJPA1VedpZFt12Ahmn4CycOmile_uSahFZnJPn_5KPISF5aCYs$
> >> [3] 
> >> https://urldefense.com/v3/__https://github.com/t-kalinowski/RFI__;!!DZ3fjg!s1-ihrZ9DPUtXpxdIpJPA1VedpZFt12Ahmn4CycOmile_uSahFZnJPn_5KPIbwXmXqY$
> >>
> >> Tomasz Kalinowski
> >>
> >>
> >>
> >> On Tue, Dec 22, 2020 at 7:24 PM Balasubramanian Narasimhan
> >>  wrote:
> >>> To deal with such Fortran issues in several packages I deal with, I
> >>> wrote the SUtools package 
> >>> (https://urldefense.com/v3/__https://github.com/bnaras/SUtools__;!!DZ3fjg!s1-ihrZ9DPUtXpxdIpJPA1VedpZFt12Ahmn4CycOmile_uSahFZnJPn_5KPIJ5BbDwA$
> >>>  ) that you
> >>> can try.  The current version generates the registration assuming
> >>> implicit Fortran naming conventions though. (I've been meaning to
> >>> upgrade it to use the gfortran -fc-prototypes-external flag which should
> >>> be easy; I might just finish that during th

Re: [Rd] From .Fortran to .Call?

[Avraham Adler]

I’ve tried recoding some of Delaporte to use the .Fortran interface and I
don’t know what I’m doing wrong but it either doesn’t work or crashes my R
instance completely.

Avi

On Sat, Dec 26, 2020 at 11:48 AM Koenker, Roger W 
wrote:

> I’ve recoded a version of one of my quantile regression fitting functions
> to use .C64 from dotCall64 rather than .Fortran.
> For a moderately large problem with n = 500,000 and p = 5, and solving
> for  1:49/50 quantiles the new version shows
> a 3% speedup, although for smaller problems it is actually slower that the
> .Fortran version.  So, I’m (provisionally)
> unimpressed by the claims that .Fortran has a big “overhead” performance
> penalty.  Compared to the(more than) an order of
> magnitude (base 10) improvement that moving from R to fortran produces,
> 3% isn’t really worth the (admittedly) minimal
> additional coding effort.
>
> > On Dec 24, 2020, at 12:39 AM, Balasubramanian Narasimhan <
> na...@stanford.edu> wrote:
> >
> > Also, just came to know about dotcall64::.C64() (on CRAN) which allows
> for Fortran to be called using .Call().
> >
> > -Naras
> >
> > On 12/23/20 8:34 AM, Balasubramanian Narasimhan wrote:
> >> I think it should be pretty easy to fix up SUtools to use the .Call
> instead of .Fortran following along the lines of
> >>
> >>
> https://urldefense.com/v3/__https://github.com/wrathematics/Romp__;!!DZ3fjg!r3_sswU4ZHCe3huoGUy2boX-Vr7aUS-RaExyeh_Rsv8gvGiABcqzvOOKZinG4kC7RtA$
> >> I too deal with a lot of f77 and so I will most likely finish it before
> the new year, if not earlier. (Would welcome testers besides myself.)
> >>
> >> Incidentally, any idea of what the performance hit is, quantitatively?
> I confess I never paid attention to it myself as most Fortran code I use
> seems pretty fast, i.e. glmnet.
> >>
> >> -Naras
> >>
> >>
> >> On 12/23/20 3:57 AM, Koenker, Roger W wrote:
> >>> Thanks to all and best wishes for a better 2021.
> >>>
> >>> Unfortunately I remain somewhat confused:
> >>>
> >>> o  Bill reveals an elegant way to get from my rudimentary
> registration setup to
> >>> one that would explicitly type the C interface functions,
> >>>
> >>> o Ivan seems to suggest that there would be no performance gain
> from doing this.
> >>>
> >>> o  Naras’s pcLasso package does use the explicit C typing, but
> then uses .Fortran
> >>> not .Call.
> >>>
> >>> o  Avi uses .Call and cites the Romp package
> https://urldefense.com/v3/__https://github.com/wrathematics/Romp__;!!DZ3fjg!r3_sswU4ZHCe3huoGUy2boX-Vr7aUS-RaExyeh_Rsv8gvGiABcqzvOOKZinG4kC7RtA$
>where it is asserted that "there is a (nearly) deprecated interface
> .Fortran() which you
> >>> should not use due to its large performance overhead.”
> >>>
> >>> As the proverbial naive R (ab)user I’m left wondering:
> >>>
> >>> o  if I updated my quantreg_init.c file in accordance with Bill’s
> suggestion could I
> >>> then simply change my .Fortran calls to .Call?
> >>>
> >>> o  and if so, do I need to include ALL the fortran subroutines in
> my src directory
> >>> or only the ones called from R?
> >>>
> >>> o  and in either case could I really expect to see a significant
> performance gain?
> >>>
> >>> Finally, perhaps I should stipulate that my fortran is strictly f77,
> so no modern features
> >>> are in play, indeed most of the code is originally written in ratfor,
> Brian Kernighan’s
> >>> dialect from ancient times at Bell Labs.
> >>>
> >>> Again,  thanks to all for any advice,
> >>> Roger
> >>>
> >>>
> >>>> On Dec 23, 2020, at 1:11 AM, Avraham Adler 
> wrote:
> >>>>
> >>>> Hello, Ivan.
> >>>>
> >>>> I used .Call instead of .Fortran in the Delaporte package [1]. What
> >>>> helped me out a lot was Drew Schmidt's Romp examples and descriptions
> >>>> [2]. If you are more comfortable with the older Fortran interface,
> >>>> Tomasz Kalinowski has a package which uses Fortran 2018 more
> >>>> efficiently [3]. I haven't tried this last in practice, however.
> >>>>
> >>>> Hope that helps,
> >>>>
> >>>> Avi
> >>>>
> >>>> [1]
> https://urldefense.com/v3/__https://CRAN.R-project.org/p

Re: [Rd] From .Fortran to .Call?

[Avraham Adler]

I had some time, so I updated a toy package I have for explaining R
and Fortran use to use both the .Call and the .Fortran interfaces [1].
I think the actual Fortran code is as close to identical as I can
reasonably make it. On my computer, the .Call interface (_f) is around
4 times as fast as the .Fortran interface (_f2).

Bill, I don't know if you can, or should, "just" change .Fortran to
.Call. You certainly cannot do the reverse. I think my source code
made as minimal a change as possible; maybe that would help.

---
set.seed(77)
A <- runif(100, 0, 2000)
microbenchmark(LLC_f(A, 500, 500), LLC_f2(A, 500, 500), times =
1L, control = list(order = 'block'), check = 'equal')
Unit: nanoseconds
expr  min   lq  mean median   uq   max neval cld
  LLC_f(A, 500, 500)  700  702  799.5906801  802  6601 1  a
 LLC_f2(A, 500, 500) 3000 3101 3328.8712   3201 3401 19802 1   b


Thanks,

Avi

[1] https://github.com/aadler/SimpFort


On Sat, Dec 26, 2020 at 10:48 PM Avraham Adler  wrote:
>
> I’ve tried recoding some of Delaporte to use the .Fortran interface and I 
> don’t know what I’m doing wrong but it either doesn’t work or crashes my R 
> instance completely.
>
> Avi
>
> On Sat, Dec 26, 2020 at 11:48 AM Koenker, Roger W  
> wrote:
>>
>> I’ve recoded a version of one of my quantile regression fitting functions to 
>> use .C64 from dotCall64 rather than .Fortran.
>> For a moderately large problem with n = 500,000 and p = 5, and solving for  
>> 1:49/50 quantiles the new version shows
>> a 3% speedup, although for smaller problems it is actually slower that the 
>> .Fortran version.  So, I’m (provisionally)
>> unimpressed by the claims that .Fortran has a big “overhead” performance 
>> penalty.  Compared to the(more than) an order of
>> magnitude (base 10) improvement that moving from R to fortran produces,  3% 
>> isn’t really worth the (admittedly) minimal
>> additional coding effort.
>>
>> > On Dec 24, 2020, at 12:39 AM, Balasubramanian Narasimhan 
>> >  wrote:
>> >
>> > Also, just came to know about dotcall64::.C64() (on CRAN) which allows for 
>> > Fortran to be called using .Call().
>> >
>> > -Naras
>> >
>> > On 12/23/20 8:34 AM, Balasubramanian Narasimhan wrote:
>> >> I think it should be pretty easy to fix up SUtools to use the .Call 
>> >> instead of .Fortran following along the lines of
>> >>
>> >> https://urldefense.com/v3/__https://github.com/wrathematics/Romp__;!!DZ3fjg!r3_sswU4ZHCe3huoGUy2boX-Vr7aUS-RaExyeh_Rsv8gvGiABcqzvOOKZinG4kC7RtA$
>> >> I too deal with a lot of f77 and so I will most likely finish it before 
>> >> the new year, if not earlier. (Would welcome testers besides myself.)
>> >>
>> >> Incidentally, any idea of what the performance hit is, quantitatively? I 
>> >> confess I never paid attention to it myself as most Fortran code I use 
>> >> seems pretty fast, i.e. glmnet.
>> >>
>> >> -Naras
>> >>
>> >>
>> >> On 12/23/20 3:57 AM, Koenker, Roger W wrote:
>> >>> Thanks to all and best wishes for a better 2021.
>> >>>
>> >>> Unfortunately I remain somewhat confused:
>> >>>
>> >>> o  Bill reveals an elegant way to get from my rudimentary 
>> >>> registration setup to
>> >>> one that would explicitly type the C interface functions,
>> >>>
>> >>> o Ivan seems to suggest that there would be no performance gain from 
>> >>> doing this.
>> >>>
>> >>> o  Naras’s pcLasso package does use the explicit C typing, but then 
>> >>> uses .Fortran
>> >>> not .Call.
>> >>>
>> >>> o  Avi uses .Call and cites the Romp package 
>> >>> https://urldefense.com/v3/__https://github.com/wrathematics/Romp__;!!DZ3fjg!r3_sswU4ZHCe3huoGUy2boX-Vr7aUS-RaExyeh_Rsv8gvGiABcqzvOOKZinG4kC7RtA$
>> >>>  where it is asserted that "there is a (nearly) deprecated interface 
>> >>> .Fortran() which you
>> >>> should not use due to its large performance overhead.”
>> >>>
>> >>> As the proverbial naive R (ab)user I’m left wondering:
>> >>>
>> >>> o  if I updated my quantreg_init.c file in accordance with Bill’s 
>> >>> suggestion could I
>> >>> then simply change my .Fortran calls to .Call?
>> >>>
>> >>> o  and if so, do I need to

Re: [Rd] Faster sorting algorithm...

[Avraham Adler]

Isn’t the default method now “radix” which is the data.table sort, and
isn’t that already parallel using openmp where available?

Avi

On Mon, Mar 15, 2021 at 12:26 PM Morgan Morgan 
wrote:

> Hi,
> I am not sure if this is the right mailing list, so apologies in advance if
> it is not.
>
> I found the following link/presentation:
> https://www.r-project.org/dsc/2016/slides/ParallelSort.pdf
>
> The implementation of fsort is interesting but incomplete (not sure why?)
> and can be improved or made faster (at least 25%  I believe). I might be
> wrong but there are maybe a couple of bugs as well.
>
> My questions are:
>
> 1/ Is the R Core team interested in a faster sorting algo? (Multithread or
> even single threaded)
>
> 2/ I see an issue with the license, which is MPL-2.0, and hence not
> compatible with base R, Python and Julia. Is there an interest to change
> the license of fsort so all 3 languages (and all the people using these
> languages) can benefit from it? (Like suggested on the first page)
>
> Please let me know if there is an interest to address the above points, I
> would be happy to look into it (free of charge of course!).
>
> Thank you
> Best regards
> Morgan
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem adding gdb to RTOOLS40 on Windows

[Avraham Adler]

Also, packages should be installed in the MSYS2 environment and not the
MINGW64. Invoke msys2.exe, synchronize with Jeroen’s via the pacman -Syu,
and then try the install. It should work as Jeroen has gdb in the upstream
repository [
https://github.com/r-windows/rtools-packages/tree/master/mingw-w64-gdb]

Thanks,

Avi

On Thu, Apr 15, 2021 at 2:59 AM Voeten, C.C. 
wrote:

> Hi Mark,
>
> Try pacman -Suy first to update pacman's package database. It's quite
> possible that the versions pacman is trying to install are no longer
> available due to being out of date.
>
> HTH,
> Cesko
>
> -Original Message-
> From: R-devel  On Behalf Of Bravington,
> Mark (Data61, Hobart)
> Sent: Thursday, April 15, 2021 6:53 AM
> To: R-Devel-2 
> Subject: [Rd] problem adding gdb to RTOOLS40 on Windows
>
> I've not been able to install gdb for RTOOLS40 on Windows 10. The rtools
> installer (rtools40-x64_86.exe) doesn't seem to include gdb by default, and
> when I follow the instructions for adding gdb (which I tracked down at
> https://github.com/r-windows/docs/blob/master/faq.md) this is what
> happened:
>
> 
> 
> $ pacman -S mingw-w64-x86_64-gdb
> resolving dependencies...
> looking for conflicting packages...
>
> Packages (8) mingw-w64-x86_64-expat-2.2.9-9002
>  mingw-w64-x86_64-gettext-0.19.8.1-9002
>  mingw-w64-x86_64-libsystre-1.0.1-9002
>  mingw-w64-x86_64-libtre-git-r128.6fb7206-9002
>  mingw-w64-x86_64-ncurses-6.1.20180526-9002
>  mingw-w64-x86_64-readline-8.0.001-2
>  mingw-w64-x86_64-termcap-1.3.1-9002
>  mingw-w64-x86_64-gdb-8.3.1-9500
>
> Total Download Size:3.46 MiB
> Total Installed Size:  83.77 MiB
>
> :: Proceed with installation? [Y/n] y
> :: Retrieving packages...
> error: failed retrieving file
> 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from cloud.r-project.org
> : The requested URL returned error: 404
> error: failed retrieving file
> 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from cran.r-project.org
> : The requested URL returned error: 404
> error: failed retrieving file
> 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from dl.bintray.com :
> The requested URL returned error: 404
> warning: failed to retrieve some files
> error: failed to commit transaction (failed to retrieve some files)
> Errors occurred, no packages were upgraded.
>
> Something very similar happens if I try the 32bit version.
>
> Do you have any suggestions?
>
>  - I used to have this stuff working OK with R3.6/RTOOLS35 but have not
> needed to go back to C for a while. The version of gdb in RTOOLS is 7.9.1;
> I tried copying that gdb.exe into RTOOLS40 but it just exited instantly
> when I tried to run it from there.
>
>  - NB I have absolutely no idea what is meant by msys2 or pacman or any of
> that, I'm just following instructions...
>
>
> Thanks
> Mark
>
>
>
>
> Mark Bravington
> CSIRO Marine Lab
> Hobart
> Australia
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Feature request: Change default library path on Windows

[Avraham Adler]

On Sun, Jul 25, 2021 at 5:56 PM Duncan Murdoch 
wrote:

> On 25/07/2021 9:54 a.m., Steve Haroz wrote:
> >> Shouldn't it be in one of the AppData directories?
> >
> > I asked that same question on twitter. Here was a response
> > (https://twitter.com/bmwiernik/status/1419033079495147522):
> > * But it's not for files that should be user-accessible, like a
> > library (cf. Zotero has preferences in AppData , but library files in
> > %USERPROFILE%/Zotero)
> > * So, for example, in R's case it could make sense for the core
> > packages to be installed in %APPDATA%/R/R-4.1.0/library" rather than
> > "C:/Program Files/R/R-4.1.0/library" (either is fairly common), but
> > user packages should be somewhere more accessible.
> >
> > Here is a quote from
> >
> https://docs.microsoft.com/en-us/windows/apps/design/app-settings/store-and-retrieve-app-data
> :
> > "App data is mutable data that is created and managed by a specific
> > app. It includes runtime state, app settings, user preferences,
> > reference content (such as the dictionary definitions in a dictionary
> > app), and other settings"
> > I don't think libraries fall into the categories of state or settings.
>
> I saw that link, and like you and Gabe, found it somewhat ambiguous.
>
> I don't know bmwiernik, but from the sound of it, he doesn't speak for
> Microsoft.
>
> So I would say that I still believe Microsoft doesn't give clear
> guidance for this.  Yes, %USERPROFILE%/R has some non-Microsoft
> precedents, but that's irrelevant.  This is an issue to take up with MS,
> not with R.  Let them describe *clearly* what they want, and R will
> (eventually) do it (but probably not before they've changed their clear
> guidance).
>
> Duncan Murdoch
>
> P.S. I am a former member of R Core who did Windows support.  I now
> detest that OS.  I suspect nobody who is still on R Core doesn't detest it.



I remember well and greatly appreciate your work, Dr. Murdoch. As someone
whose R work must be almost exclusively on Windows due to the nature of my
employment, I want to once again show appreciation to the members of R Core
and others, specifically Drs. Liggs and Tierney, and Jeroen Oomes, Dirk
Eddelbuttel, and the many others who continue to support R for Windows.

Avi

> --
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] svd for Large Matrix

[Avraham Adler]

If you’re crossproding X by itself, I think passing symmetric = TRUE to
eigen will eke out more speed.

Avi

On Mon, Aug 16, 2021 at 6:30 PM Radford Neal  wrote:

> > Dario Strbenac  writes:
> >
> > I have a real scenario involving 45 million biological cells
> > (samples) and 60 proteins (variables) which leads to a segmentation
> > fault for svd. I thought this might be a good example of why it
> > might benefit from a long vector upgrade.
>
> Rather than the full SVD of a 4500x60 X, my guess is that you
> may really only be interested in the eigenvalues and eigenvectors of
> X^T X, in which case eigen(t(X)%*%X) would probably be much faster.
> (And eigen(crossprod(X)) would be even faster.)
>
> Note that if you instead want the eigenvalues and eigenvectors of
> X X^T (which is an enormous matrix), the eigenvalues of this are the
> same as those of X^T X, and the eigenvectors are Xv, where v is an
> eigenvector of X^T X.
>
> For example, with R 4.0.2, and the reference BLAS/LAPACK, I get
>
>   > X<-matrix(rnorm(10),1,10)
>   > system.time(for(i in 1:1000) rs<-svd(X))
>  user  system elapsed
> 2.393   0.008   2.403
>   > system.time(for(i in 1:1000) re<-eigen(crossprod(X)))
>  user  system elapsed
> 0.609   0.000   0.609
>   > rs$d^2
>[1] 10568.003 10431.864 10318.959 10219.961 10138.025 10068.566
> 9931.538
>[8]  9813.841  9703.818  9598.532
>   > re$values
>[1] 10568.003 10431.864 10318.959 10219.961 10138.025 10068.566
> 9931.538
>[8]  9813.841  9703.818  9598.532
>
> Possibly some other LAPACK might implement svd better, though I
> suspect that R will allocate more big matrices than really necessary
> for the svd even aside from whatever LAPACK is doing.
>
> Regards,
>
>Radford Neal
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [External] Re: Update on rtools4 and ucrt support

[Avraham Adler]

On Mon, Aug 23, 2021 at 11:09 PM  wrote:

> On Mon, 23 Aug 2021, Duncan Murdoch wrote:
>
> > On 23/08/2021 8:15 a.m., jan Vitek via R-devel wrote:
> >> Hi Jeroen,
> >>
> >> I mostly lurk on this list, but I was struck by your combative tone.
> >>
> >> To pick on two random bits:
> >>
> >>> … a 6gb tarball with manually built things on his personal machine…
> >>
> >>> … a black-box system that is so opaque and complex that only one person
> >>> knows how it works, and would make it much more difficult for
> >>> students, universities, and other organisations to build R packages
> >>> and libraries on Windows…
> >>
> >>
> >> Tomas’ tool chain isn't a blackbox, it has copious documentation (see
> [1])
> >> and builds on any machine thanks to the provided docker container.
> >>
> >> This is not to criticise your work which has its unique strengths, but
> to
> >> state the obvious: these strengths are best discussed without passion
> >> based on factually accurate descriptions.
> >
> > I agree with Jan.  I'm not sure a discussion in this forum would be
> fruitful,
> > but I really wish Jeroen and Tomas would get together, aiming to merge
> their
> > toolchains, keeping the best aspects of both.
> >
> > I haven't been involved in the development of either one, but have been
> a
> > "victim" of the two chain rivalry, because the rgl package is not easy
> to
> > build.  I get instructions from each of them on how to do the build, and
> > those instructions for one toolchain generally break the build on the
> other
> > one.  While it is probably possible to detect the toolchain and have the
> > build adapt to whichever one is in use, it would be a lot easier for me
> (and
> > I imagine every other maintainer of a package using external libs) if I
> just
> > had to follow one set of instructions.
> >
> > Duncan Murdoch
>
> Here are just a few comments from my perspective (I am an R-core
> member, but am not part of the CRAN team and do only very limited work
> on Windows). Other R-core members may have different perspectives and
> insights.
>
> One bit of background: dealing with encoding issues on Windows has
> been taking an unsustainable amount of R-core resources for some time
> now. Tomas Kalibera has been taking the lead on trying to address
> these issues in the existing framework, but this means he has not had
> the time to make any of the many other valuable and important
> contributions he could make. The only viable way forward is to move to
> a Windows tool chain that supports UTF-8 as the C library current
> encoding via the Windows UCRT framework.
>
> Tomas Kalibera has, on behalf of all of R core and in
> coordination with CRAN, been looking for a way forward for some
> time and has reported on the progress in several blog posts at
> https://developer.r-project.org/Blog/public/. This has lead to
> the development of the MXE-based UCRT tool chain, which is now
> well tested and ready for deployment.  Checks using the UCRT tool
> chain have been part of the CRAN check process for a while. I
> believe CRAN plans to switch R-devel checks and builds to the
> UCRT tool chain during the upcoming CRAN downtime. I expect there
> will be some communication from CRAN on this soon, including on
> any issues in supporting binaries for both R-devel and R-patched.
>
> In putting together something as large as a tool chain there will
> always be many choices, each with advantages and disadvantages.  Some
> things may be advantages in some settings and not others. Taking just
> one case in point: Cross compilation. This is likely to be a better
> approach for CRAN in the future and is supported by the MXE framework
> on which the new tool chain is based.
>
> The much more recent changes in rtools4 to support UCRT are at this
> point not yet as well tested as the new tool chain. Once these changes
> to rtools4 mature, and if binary compatibility can be assured, then
> having a second tool chain may be useful in some cases.  But if there
> are incompatibilities then it will be up to rtools4 to keep up with
> the tool chain used by CRAN. On the other, contributing to improving
> the MXE-based tool chain may be a better investment of time.
>
> Best,
>
> luke
>
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa  Phone: 319-335-3386
> Department of Statistics andFax:   319-335-3017
> Actuarial Science
> 241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
> Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
>

Thank you, Dr. Tierney. However, I am concerned about the not-insignificant
number of us who for various reasons can only do our development on
Windows. Rtools has been the official tool chain with which to build
windows for the at least 20 year

Re: [Rd] [External] Re: Update on rtools4 and ucrt support

[Avraham Adler]

On Tue, Aug 24, 2021 at 12:47 AM Simon Urbanek 
wrote:

> Avi,
>
> please see the announcement:
>
>
> https://developer.r-project.org/Blog/public/2021/03/12/windows/utf-8-toolchain-and-cran-package-checks/index.html
>
> the documentation is in
>
>
> https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/ucrt3/howto.html
>
> Cheers,
> Simon
>
>
>
>
> On Aug 24, 2021, at 8:34 AM, Avraham Adler 
> wrote:
>
> On Mon, Aug 23, 2021 at 11:09 PM  wrote:
>
> On Mon, 23 Aug 2021, Duncan Murdoch wrote:
>
> On 23/08/2021 8:15 a.m., jan Vitek via R-devel wrote:
>
> Hi Jeroen,
>
> I mostly lurk on this list, but I was struck by your combative tone.
>
> To pick on two random bits:
>
> … a 6gb tarball with manually built things on his personal machine…
>
>
> … a black-box system that is so opaque and complex that only one person
> knows how it works, and would make it much more difficult for
> students, universities, and other organisations to build R packages
> and libraries on Windows…
>
>
>
> Tomas’ tool chain isn't a blackbox, it has copious documentation (see
>
> [1])
>
> and builds on any machine thanks to the provided docker container.
>
> This is not to criticise your work which has its unique strengths, but
>
> to
>
> state the obvious: these strengths are best discussed without passion
> based on factually accurate descriptions.
>
>
> I agree with Jan.  I'm not sure a discussion in this forum would be
>
> fruitful,
>
> but I really wish Jeroen and Tomas would get together, aiming to merge
>
> their
>
> toolchains, keeping the best aspects of both.
>
> I haven't been involved in the development of either one, but have been
>
> a
>
> "victim" of the two chain rivalry, because the rgl package is not easy
>
> to
>
> build.  I get instructions from each of them on how to do the build, and
> those instructions for one toolchain generally break the build on the
>
> other
>
> one.  While it is probably possible to detect the toolchain and have the
> build adapt to whichever one is in use, it would be a lot easier for me
>
> (and
>
> I imagine every other maintainer of a package using external libs) if I
>
> just
>
> had to follow one set of instructions.
>
> Duncan Murdoch
>
>
> Here are just a few comments from my perspective (I am an R-core
> member, but am not part of the CRAN team and do only very limited work
> on Windows). Other R-core members may have different perspectives and
> insights.
>
> One bit of background: dealing with encoding issues on Windows has
> been taking an unsustainable amount of R-core resources for some time
> now. Tomas Kalibera has been taking the lead on trying to address
> these issues in the existing framework, but this means he has not had
> the time to make any of the many other valuable and important
> contributions he could make. The only viable way forward is to move to
> a Windows tool chain that supports UTF-8 as the C library current
> encoding via the Windows UCRT framework.
>
> Tomas Kalibera has, on behalf of all of R core and in
> coordination with CRAN, been looking for a way forward for some
> time and has reported on the progress in several blog posts at
> https://developer.r-project.org/Blog/public/. This has lead to
> the development of the MXE-based UCRT tool chain, which is now
> well tested and ready for deployment.  Checks using the UCRT tool
> chain have been part of the CRAN check process for a while. I
> believe CRAN plans to switch R-devel checks and builds to the
> UCRT tool chain during the upcoming CRAN downtime. I expect there
> will be some communication from CRAN on this soon, including on
> any issues in supporting binaries for both R-devel and R-patched.
>
> In putting together something as large as a tool chain there will
> always be many choices, each with advantages and disadvantages.  Some
> things may be advantages in some settings and not others. Taking just
> one case in point: Cross compilation. This is likely to be a better
> approach for CRAN in the future and is supported by the MXE framework
> on which the new tool chain is based.
>
> The much more recent changes in rtools4 to support UCRT are at this
> point not yet as well tested as the new tool chain. Once these changes
> to rtools4 mature, and if binary compatibility can be assured, then
> having a second tool chain may be useful in some cases.  But if there
> are incompatibilities then it will be up to rtools4 to keep up with
> the tool chain used by CRAN. On the other, contributing to improving
> the MXE-based tool chain may be a better investment 

Re: [Rd] WISH: set.seed(seed) to produce error if length(seed) != 1 (now silent)

[Avraham Adler]

Hi, Henrik.

I’m curious, other than proper programming practice, why?

Avi

On Fri, Sep 17, 2021 at 11:48 AM Henrik Bengtsson <
henrik.bengts...@gmail.com> wrote:

> Hi,
>
> according to help("set.seed"), argument 'seed' to set.seed() should be:
>
>   a single value, interpreted as an integer, or NULL (see ‘Details’).
>
> From code inspection (src/main/RNG.c) and testing, it turns out that
> if you pass a 'seed' with length greater than one, it silently uses
> seed[1], e.g.
>
> > set.seed(1); sum(.Random.seed)
> [1] 4070365163
> > set.seed(1:3); sum(.Random.seed)
> [1] 4070365163
> > set.seed(1:100); sum(.Random.seed)
> [1] 4070365163
>
> I'd like to suggest that set.seed() produces an error if length(seed)
> > 1.  As a reference, for length(seed) == 0, we get:
>
> > set.seed(integer(0))
> Error in set.seed(integer(0)) : supplied seed is not a valid integer
>
> /Henrik
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] R-devel (r81196) hanging at dchisq(large) (PR#13309)

[Avraham Adler]

I am building r-devel on Windows 10 64bit using Jeroen's mingw system,
and I am finding that my make check-devel hangs on the above issue.
Everything is vanila except that I am using OpenBLAS 0.3.18. I have
been using OpenBLAS for over a decade and have not had this issue
before. Is there anything I can do to dig deeper into this issue from
my end? Could there be anything that changed in R-devel that may have
triggered this? The bugzilla report doesn't have any code attached to
it.

Thank you,

Avi

sessionInfo:
R Under development (unstable) (2021-11-15 r81196)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19043)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_4.2.0 tools_4.2.0

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R-devel (r81196) hanging at dchisq(large) (PR#13309)

[Avraham Adler]

On Tue, Nov 16, 2021 at 8:43 AM Martin Maechler
 wrote:
>
> >>>>> Avraham Adler
> >>>>> on Tue, 16 Nov 2021 02:35:56 + writes:
>
> > I am building r-devel on Windows 10 64bit using Jeroen's mingw system,
> > and I am finding that my make check-devel hangs on the above issue.
> > Everything is vanila except that I am using OpenBLAS 0.3.18. I have
> > been using OpenBLAS for over a decade and have not had this issue
> > before. Is there anything I can do to dig deeper into this issue from
> > my end? Could there be anything that changed in R-devel that may have
> > triggered this? The bugzilla report doesn't have any code attached to
> > it.
>
> > Thank you,
> > Avi
>
> Hmm.. it would've be nice to tell a bit more, instead of having all
> your readers to search links, etc.
>
> In the bugzilla bug report PR#13309
> https://bugs.r-project.org/show_bug.cgi?id=13309 ,the example was
>
>  dchisq(x=Inf, df=10, ncp=1)
>
> I had fixed the bug 13 years ago, in svn rev 47005
> with regression test in /tests/d-p-q-r-tests.R :
>
>
> ## Non-central Chi^2 density for large x
> stopifnot(0 == dchisq(c(Inf, 1e80, 1e50, 1e40), df=10, ncp=1))
> ## did hang in 2.8.0 and earlier (PR#13309).
>
>
> and you are seeing your version of R hanging at exactly this
> location?
>
>
> I'd bet quite a bit that the underlying code in these
> non-central chi square computations *never* calls BLAS and hence
> I cannot imagine how openBLAS could play a role.
>
> However, there must be something peculiar in your compiler setup,
> compilation options, 
> as of course the above regression test has been run 100s of
> 1000s of times also under Windows in the last 13 years ..
>
> Last but not least (but really only vaguely related):
>There is still a FIXME in the source code (but not about
> hanging, but rather of loosing some accuracy in border cases),
> see e.g. https://svn.r-project.org/R/trunk/src/nmath/dnchisq.c
> and for that reason I had written an R version of that C code
> even back in 2008 which I've made available in  CRAN package
> DPQ  a few years ago (together with many other D/P/Q
> distribution computations/approximations).
>  -> https://cran.r-project.org/package=DPQ
>
> Best,
> Martin
>

Hello, Martin.

Apologies, I thought the PR # was sufficient. Yes, I am seeing this at
this exact location. This is what I saw in d-p-q-r-tst-2.Rout.fail and
I then ran d-p-q-r-tst.R line-by-line and R hung precisely after
`stopifnot(0 == dchisq(c(Inf, 1e80, 1e50, 1e40), df=10, ncp=1))`.

Is it at all possible that this has to do with the recent change from
bd0 to ebd0 (PR #15628) [1]?

For completeness, I ran all the code _beneath_ the call, and while
nothing else cause an infinite loop, I posted what I believe may be
unexpected results below,

Thank you,

Avi

[1]: https://bugs.r-project.org/show_bug.cgi?id=15628

> ## FIXME ?!:  MxM/2 seems +- ok ??
> (dLmM <- dnbinom(xL, mu = 1, size = MxM))  # all NaN but the last
Warning in dnbinom(xL, mu = 1, size = MxM) : NaNs produced
 [1] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN   0
> (dLpI <- dnbinom(xL, prob=1/2, size = Inf))#  ditto
Warning in dnbinom(xL, prob = 1/2, size = Inf) : NaNs produced
 [1] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN   0
> (dLpM <- dnbinom(xL, prob=1/2, size = MxM))#  ditto
Warning in dnbinom(xL, prob = 1/2, size = MxM) : NaNs produced
 [1] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN   0

> d <- dnbinom(x,  mu = mu, size = Inf) # gave NaN (for 0 and L), now all 0
Warning in dnbinom(x, mu = mu, size = Inf) : NaNs produced
> p <- pnbinom(x,  mu = mu, size = Inf) # gave all NaN, now uses ppois(x, mu)
Warning in pnbinom(x, mu = mu, size = Inf) : NaNs produced

> pp <- (0:16)/16
> q <- qnbinom(pp, mu = mu, size = Inf) # gave all NaN
> set.seed(1); NI <- rnbinom(32, mu = mu, size = Inf)# gave all NaN
> set.seed(1); N2 <- rnbinom(32, mu = mu, size = L  )
> stopifnot(exprs = {
+ all.equal(d, c(0.006737947, 0.033689735, 0.0842243375,
0.140373896, 0,0,0,0), tol = 9e-9)# 7.6e-10
+ all.equal(p, c(0.006737947, 0.040427682, 0.1246520195,
0.265025915, 1,1,1,1), tol = 9e-9)# 7.3e-10
+ all.equal(d, dpois(x, mu))# current implementation: even identical()
+ all.equal(p, ppois(x, mu))
+ q == c(0, 2, 3, 3, 3, 4, 4, 4, 5, 5, 6, 6, 6, 7, 8, 9, Inf)
+ q == qpois(pp, mu)
+ identical(NI, N2)
+ })
Error: d and c(0.006737947, 0.033689735, 0.0842243375, 0.140373896, 0,
0,   are not equal:
  &#

Re: [Rd] R-devel (r81196) hanging at dchisq(large) (PR#13309)

[Avraham Adler]

On Tue, Nov 16, 2021 at 10:42 PM Kevin Ushey  wrote:
>
> Do you see this same hang in a build of R with debug symbols? Can you
> try running R with GDB, or even WinDbg or another debugger, to see
> what the call stack looks like when the hang occurs? Does the hang
> depend on the number of threads used by OpenBLAS?
>
> On the off chance it's relevant, I've seen hangs / crashes when using
> a multithreaded OpenBLAS with R on some Linux systems before, but
> never found the time to isolate a root cause.
>


This last was a good thought, Kevin, as I had just compiled OpenBLAS
3.18 multi-threaded, but I recompiled it single threaded and it still
crashes. The version of R I built from source last time, (2021-05-20
r80347), does not hang when calling `dchisq(c(Inf, 1e80, 1e50, 1e40),
df=10, ncp=1)`. I think I built that with OpenBLAS 3.15. I can try
doing that here. As for building with debug symbols, I have never done
that before, so if you could provide some guidance (off-list if you
think it is inappropriate to keep it here) or point me in the
direction of some already posted advice, I would appreciate it!

Avi



> Best,
> Kevin
>
> On Tue, Nov 16, 2021 at 5:12 AM Avraham Adler  wrote:
> >
> > On Tue, Nov 16, 2021 at 8:43 AM Martin Maechler
> >  wrote:
> > >
> > > >>>>> Avraham Adler
> > > >>>>> on Tue, 16 Nov 2021 02:35:56 + writes:
> > >
> > > > I am building r-devel on Windows 10 64bit using Jeroen's mingw 
> > > system,
> > > > and I am finding that my make check-devel hangs on the above issue.
> > > > Everything is vanila except that I am using OpenBLAS 0.3.18. I have
> > > > been using OpenBLAS for over a decade and have not had this issue
> > > > before. Is there anything I can do to dig deeper into this issue 
> > > from
> > > > my end? Could there be anything that changed in R-devel that may 
> > > have
> > > > triggered this? The bugzilla report doesn't have any code attached 
> > > to
> > > > it.
> > >
> > > > Thank you,
> > > > Avi
> > >
> > > Hmm.. it would've be nice to tell a bit more, instead of having all
> > > your readers to search links, etc.
> > >
> > > In the bugzilla bug report PR#13309
> > > https://bugs.r-project.org/show_bug.cgi?id=13309 ,the example was
> > >
> > >  dchisq(x=Inf, df=10, ncp=1)
> > >
> > > I had fixed the bug 13 years ago, in svn rev 47005
> > > with regression test in /tests/d-p-q-r-tests.R :
> > >
> > >
> > > ## Non-central Chi^2 density for large x
> > > stopifnot(0 == dchisq(c(Inf, 1e80, 1e50, 1e40), df=10, ncp=1))
> > > ## did hang in 2.8.0 and earlier (PR#13309).
> > >
> > >
> > > and you are seeing your version of R hanging at exactly this
> > > location?
> > >
> > >
> > > I'd bet quite a bit that the underlying code in these
> > > non-central chi square computations *never* calls BLAS and hence
> > > I cannot imagine how openBLAS could play a role.
> > >
> > > However, there must be something peculiar in your compiler setup,
> > > compilation options, 
> > > as of course the above regression test has been run 100s of
> > > 1000s of times also under Windows in the last 13 years ..
> > >
> > > Last but not least (but really only vaguely related):
> > >There is still a FIXME in the source code (but not about
> > > hanging, but rather of loosing some accuracy in border cases),
> > > see e.g. https://svn.r-project.org/R/trunk/src/nmath/dnchisq.c
> > > and for that reason I had written an R version of that C code
> > > even back in 2008 which I've made available in  CRAN package
> > > DPQ  a few years ago (together with many other D/P/Q
> > > distribution computations/approximations).
> > >  -> https://cran.r-project.org/package=DPQ
> > >
> > > Best,
> > > Martin
> > >
> >
> > Hello, Martin.
> >
> > Apologies, I thought the PR # was sufficient. Yes, I am seeing this at
> > this exact location. This is what I saw in d-p-q-r-tst-2.Rout.fail and
> > I then ran d-p-q-r-tst.R line-by-line and R hung precisely after
> > `stopifnot(0 == dchisq(c(Inf, 1e80, 1e50, 1e40), df=10, ncp=1))`.
> >
> > Is it at all possible that this has to do with the recent change from
> > bd0 to ebd0 (PR #15628) [1]?
> >
> > For completeness, I

Re: [Rd] R-devel (r81196) hanging at dchisq(large) (PR#13309)

[Avraham Adler]

Hello, Martin et. al.

I apologize for top posting, but I believe I have tracked down the
difference why last time my build worked and now it hangs on
`dchisq(c(Inf, 1e80, 1e50, 1e40), df=10, ncp=1)`. and it's NOT the
BLAS. I built against both 3.15 AND R's vanilla and it hung both
times. The issue was passing "march=skylake". I own an i7-8700K which
gcc considers a skylake. When I pass mtune=skylake, it does not hang
and the make check-devel after the build completes.

Below is a list of the different flags passed when using mtune vs.
march. It stands to reason that at least one of them contributed to
the hanging issue which Martin fixed in
https://bugs.r-project.org/show_bug.cgi?id=13309. While I recognize
the obvious ones, I'm not an expert and do not understand which if any
may be the culprit. For reference, most of these flags are described
here: https://gcc.gnu.org/onlinedocs/gcc-8.3.0/gcc/x86-Options.html#x86-Options.

All the following flags are DISABLED for mtune=skylake (so
march=x86-64) and ENABLED when passing march-skylake. For the record,
I usually passed march in the past without problem:

madx, maes, mavx, mavx2, mbmi, mbmi2, mclflushopt, mcx16, mf16c, mfma,
mfsgsbase, mhle, mlzcnt, mmovbe, mpclmul, mpopcnt, mprfchw, mrdrnd,
mrdseed, msahf, msgx, msse3, msse4, msse4.1, msse4.2, mssse3, mxsave,
mxsavec, mxsaveopt, and mxsaves.

Inversely, mno-sse4 is enabled when using mtune and disabled when
using arch, of course.

For completeness, the following two are disabled on both mtune and
march but enabled when passing march=native, otherwise the latter is
the same as march=skylake: mabm and mrtm. Obviously these cannot
contribute to the hanging issue.

Any ideas, especially from the experts who understand how the flags
would address the code in dchisq, would be greatly appreciated.

Thank you,

Avi

On Wed, Nov 17, 2021 at 7:15 AM Avraham Adler  wrote:
>
> On Tue, Nov 16, 2021 at 10:42 PM Kevin Ushey  wrote:
> >
> > Do you see this same hang in a build of R with debug symbols? Can you
> > try running R with GDB, or even WinDbg or another debugger, to see
> > what the call stack looks like when the hang occurs? Does the hang
> > depend on the number of threads used by OpenBLAS?
> >
> > On the off chance it's relevant, I've seen hangs / crashes when using
> > a multithreaded OpenBLAS with R on some Linux systems before, but
> > never found the time to isolate a root cause.
> >
>
>
> This last was a good thought, Kevin, as I had just compiled OpenBLAS
> 3.18 multi-threaded, but I recompiled it single threaded and it still
> crashes. The version of R I built from source last time, (2021-05-20
> r80347), does not hang when calling `dchisq(c(Inf, 1e80, 1e50, 1e40),
> df=10, ncp=1)`. I think I built that with OpenBLAS 3.15. I can try
> doing that here. As for building with debug symbols, I have never done
> that before, so if you could provide some guidance (off-list if you
> think it is inappropriate to keep it here) or point me in the
> direction of some already posted advice, I would appreciate it!
>
> Avi
>
>
>
> > Best,
> > Kevin
> >
> > On Tue, Nov 16, 2021 at 5:12 AM Avraham Adler  
> > wrote:
> > >
> > > On Tue, Nov 16, 2021 at 8:43 AM Martin Maechler
> > >  wrote:
> > > >
> > > > >>>>> Avraham Adler
> > > > >>>>> on Tue, 16 Nov 2021 02:35:56 + writes:
> > > >
> > > > > I am building r-devel on Windows 10 64bit using Jeroen's mingw 
> > > > system,
> > > > > and I am finding that my make check-devel hangs on the above 
> > > > issue.
> > > > > Everything is vanila except that I am using OpenBLAS 0.3.18. I 
> > > > have
> > > > > been using OpenBLAS for over a decade and have not had this issue
> > > > > before. Is there anything I can do to dig deeper into this issue 
> > > > from
> > > > > my end? Could there be anything that changed in R-devel that may 
> > > > have
> > > > > triggered this? The bugzilla report doesn't have any code 
> > > > attached to
> > > > > it.
> > > >
> > > > > Thank you,
> > > > > Avi
> > > >
> > > > Hmm.. it would've be nice to tell a bit more, instead of having all
> > > > your readers to search links, etc.
> > > >
> > > > In the bugzilla bug report PR#13309
> > > > https://bugs.r-project.org/show_bug.cgi?id=13309 ,the example was
> > > >
> > > >  dchisq(x=Inf, df=10, ncp=1)
> > > >
> 

Re: [Rd] R-devel (r81196) hanging at dchisq(large) (PR#13309)

[Avraham Adler]

Hello.

I have isolated the issue: it is the fused-multiply-add instruction
set (FMA on Intel processors). Running -march=skylake -mno-fma not
only does not hang, but passes make check-all (using R's native BLAS).
My intuition remains that something in the new more precise ebd0 code
used in dpois_raw—called by dgamma, called by dchsq, called by
dnchisq—is hanging when the assembler uses FMA. Unfortunately, I have
come across other cases online where the extra precision and the
different assembler code of FMA vs. non-FMA has caused bugs, such as
[1]. Page 5 of this paper by Dr. William Kahan sheds some light on why
this may be happening [2] (PDF).

Martin & Morton, having written (PR#15628 [3]) and/or implemented the
ebd0 code that is now being used, can either of you think of any
reason why it would hang if compiled using FMA? Again, I'm not a
professional, but line 325 of the ebd0 function in bd0.c [4] has
"ADD1(-x * log1pmx ((M * fg - x) / x))" which looks like a
Multiply-Add to me, at least in the inner parenthesis. Is there
anything that can be, or should be, done with the code to prevent the
hang, or must we forbid the use of FMA instructions (and I guess FMA4
on AMD processors) when compiling R?

Also, What happens in the case where M/x neither over- nor
under-flowed, M_LN2 * ((double) -e) <= 1. + DBL_MAX / x, fg != 1, and
after 4 loops of lines 329 & 330, *yh is still finite? How does ebd0
exit in that case? There is no "return" after line 331. Am I missing
something? Could that be related to this issue?

As an aside, could ebd0 be enhanced by using FMA instructions on
processors which support them?

Thank you very much,

Avi

[1] 
https://flameeyes.blog/2014/10/27/the-subtlety-of-modern-cpus-or-the-search-for-the-phantom-bug/
[2] https://people.eecs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF
[3] https://bugs.r-project.org/show_bug.cgi?id=15628
[4] https://github.com/wch/r-source/blob/trunk/src/nmath/bd0.c

On Wed, Nov 17, 2021 at 3:55 PM Avraham Adler  wrote:
>
> Hello, Martin et. al.
>
> I apologize for top posting, but I believe I have tracked down the
> difference why last time my build worked and now it hangs on
> `dchisq(c(Inf, 1e80, 1e50, 1e40), df=10, ncp=1)`. and it's NOT the
> BLAS. I built against both 3.15 AND R's vanilla and it hung both
> times. The issue was passing "march=skylake". I own an i7-8700K which
> gcc considers a skylake. When I pass mtune=skylake, it does not hang
> and the make check-devel after the build completes.
>
> Below is a list of the different flags passed when using mtune vs.
> march. It stands to reason that at least one of them contributed to
> the hanging issue which Martin fixed in
> https://bugs.r-project.org/show_bug.cgi?id=13309. While I recognize
> the obvious ones, I'm not an expert and do not understand which if any
> may be the culprit. For reference, most of these flags are described
> here: 
> https://gcc.gnu.org/onlinedocs/gcc-8.3.0/gcc/x86-Options.html#x86-Options.
>
> All the following flags are DISABLED for mtune=skylake (so
> march=x86-64) and ENABLED when passing march-skylake. For the record,
> I usually passed march in the past without problem:
>
> madx, maes, mavx, mavx2, mbmi, mbmi2, mclflushopt, mcx16, mf16c, mfma,
> mfsgsbase, mhle, mlzcnt, mmovbe, mpclmul, mpopcnt, mprfchw, mrdrnd,
> mrdseed, msahf, msgx, msse3, msse4, msse4.1, msse4.2, mssse3, mxsave,
> mxsavec, mxsaveopt, and mxsaves.
>
> Inversely, mno-sse4 is enabled when using mtune and disabled when
> using arch, of course.
>
> For completeness, the following two are disabled on both mtune and
> march but enabled when passing march=native, otherwise the latter is
> the same as march=skylake: mabm and mrtm. Obviously these cannot
> contribute to the hanging issue.
>
> Any ideas, especially from the experts who understand how the flags
> would address the code in dchisq, would be greatly appreciated.
>
> Thank you,
>
> Avi
>
> On Wed, Nov 17, 2021 at 7:15 AM Avraham Adler  wrote:
> >
> > On Tue, Nov 16, 2021 at 10:42 PM Kevin Ushey  wrote:
> > >
> > > Do you see this same hang in a build of R with debug symbols? Can you
> > > try running R with GDB, or even WinDbg or another debugger, to see
> > > what the call stack looks like when the hang occurs? Does the hang
> > > depend on the number of threads used by OpenBLAS?
> > >
> > > On the off chance it's relevant, I've seen hangs / crashes when using
> > > a multithreaded OpenBLAS with R on some Linux systems before, but
> > > never found the time to isolate a root cause.
> > >
> >
> >
> > This last was a good thought, Kevin, as I had just compiled OpenBLAS
> > 3.18 multi-threaded, but I recompi

Re: [Rd] R-devel (r81196) hanging at dchisq(large) (PR#13309)

[Avraham Adler]

On Wed, Nov 24, 2021 at 11:45 AM Martin Maechler
 wrote:
>
> >>>>> Avraham Adler  on Thu, 18 Nov 2021 02:18:54 + writes:
>
> > Hello.  I have isolated the issue: it is the
> > fused-multiply-add instruction set (FMA on Intel
> > processors). Running -march=skylake -mno-fma not only does
> > not hang, but passes make check-all (using R's native
> > BLAS).  My intuition remains that something in the new
> > more precise ebd0 code used in dpois_raw—called by dgamma,
> > called by dchsq, called by dnchisq—is hanging when the
> > assembler uses FMA. Unfortunately, I have come across
> > other cases online where the extra precision and the
> > different assembler code of FMA vs. non-FMA has caused
> > bugs, such as [1]. Page 5 of this paper by Dr. William
> > Kahan sheds some light on why this may be happening [2]
> > (PDF).
>
> > Martin & Morton, having written (PR#15628 [3]) and/or
> > implemented the ebd0 code that is now being used, can
> > either of you think of any reason why it would hang if
> > compiled using FMA?
>
> I vaguely remember I had a version of ebd0(), either Morton
> Welinder's original, or a slight modification of it that needed some
> mending, because in some border case, there was an out of
> array-boundary indexing... but that's just a vague recollection.
>
> I had investigated  ebd0()'s behavior quite a bit, also notably
> the version -- together with a pure R code version --
> in my CRAN package DPQ, yesterday updated to version 0.5-0 on CRAN
> {written in Summer, but published to CRAN only yesterday}
> where I have  dpois_raw() optionally using several experimental versions of
> bd0(), and both 'pure R' and a C version of ebd0(),
> as DPQ::ebd0() and DPQ::edb0C()
> each with an option  'verbose' which shows you which branches are chosen
> for the given arguments.
>
> So, if you install this version (0.5-0 or newer) from the development
> sources, using the *same* FMA configuration,
> I hope you should see the same "hanging" but would be able to see some
> more.. ?
>
> Can you install it from R-forge
>
> install.packages("DPQ", type = "source",
>  repos="http://R-Forge.R-project.org";)
>
> and then experiment?
> I'd be grateful  {and we maybe can move "off - mailing list"}
>
> Thank you in advance,
> Martin
>
> Martin Maechler
> ETH Zurich  and  R Core team

Sure. Responding here simply for closure. Will direct further
questions and output directly to you.

Thank yyou,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] string concatenation operator (revisited)

[Avraham Adler]

Gabe, I agree that missingness is important to factor in. To somewhat abuse
the terminology, NA is often used to represent missingness. Perhaps
concatenating character something with character something missing should
result in the original character?

Avi

On Mon, Dec 6, 2021 at 3:35 PM Gabriel Becker  wrote:

> Hi All,
>
> Seeing this and the other thread (and admittedly not having clicked through
> to the linked r-help thread), I wonder about NAs.
>
> Should NA  "hi there"  not result in NA_character_? This is not
> what any of the paste functions do, but in my opinoin, NA + 
> seems like it should be NA  (not "NA"), particularly if we are talking
> about `+` overloading, but potentially even in the case of a distinct
> concatenation operator?
>
> I guess what I'm saying is that in my head missingness propagation rules
> should take priority in such an operator (ie NA +  should
> *always * be NA).
>
> Is that something others disagree with, or has it just not come up yet in
> (the parts I have read) of this discussion?
>
> Best,
> ~G
>
> On Mon, Dec 6, 2021 at 10:03 AM Radford Neal 
> wrote:
>
> > > > In pqR (see pqR-project.org), I have implemented ! and !! as binary
> > > > string concatenation operators, equivalent to paste0 and paste,
> > > > respectively.
> > > >
> > > > For instance,
> > > >
> > > >  > "hello" ! "world"
> > > >  [1] "helloworld"
> > > >  > "hello" !! "world"
> > > >  [1] "hello world"
> > > >  > "hello" !! 1:4
> > > >  [1] "hello 1" "hello 2" "hello 3" "hello 4"
> > >
> > > I'm curious about the details:
> > >
> > > Would `1 ! 2` convert both to strings?
> >
> > They're equivalent to paste0 and paste, so 1 ! 2 produces "12", just
> > like paste0(1,2) does.  Of course, they wouldn't have to be exactly
> > equivalent to paste0 and paste - one could impose stricter
> > requirements if that seemed better for error detection.  Off hand,
> > though, I think automatically converting is more in keeping with the
> > rest of R.  Explicitly converting with as.character could be tedious.
> >
> > I suppose disallowing logical arguments might make sense to guard
> > against typos where ! was meant to be the unary-not operator, but
> > ended up being a binary operator, after some sort of typo.  I doubt
> > that this would be a common error, though.
> >
> > (Note that there's no ambiguity when there are no typos, except that
> > when negation is involved a space may be needed - so, for example,
> > "x" !  !TRUE is "xFALSE", but "x"!!TRUE is "x TRUE".  Existing uses of
> > double negation are still fine - eg, a <- !!TRUE still sets a to TRUE.
> > Parsing of operators is greedy, so "x"!!!TRUE is "x FALSE", not "xTRUE".)
> >
> > > Where does the binary ! fit in the operator priority?  E.g. how is
> > >
> > >   a ! b > c
> > >
> > > parsed?
> >
> > As (a ! b) > c.
> >
> > Their precedence is between that of + and - and that of < and >.
> > So "x" ! 1+2 evalates to "x3" and "x" ! 1+2 < "x4" is TRUE.
> >
> > (Actually, pqR also has a .. operator that fixes the problems with
> > generating sequences with the : operator, and it has precedence lower
> > than + and - and higher than ! and !!, but that's not relevant if you
> > don't have the .. operator.)
> >
> >Radford Neal
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Improved LP/MIP solver

[Avraham Adler]

On Sun, Dec 12, 2021 at 3:44 PM Julian Hall  wrote:
>
> Dear All,
>
> I am leading the development of HiGHS, which is now the top performing open 
> source linear optimization software on the industry standard benchmarks. In 
> particular, our MIP solver out-performs SCIP, and is way ahead of the COIN-OR 
> solver Cbc.
>
> HiGHS solves LPs via simplex or interior point, MIPs via branch-and-cut, and 
> QPs via an active set method.
>
> We were wondering what interest there would be in developing an R interface 
> to HiGHS. I'm not an R user, but have done a bit of searching and see 
> references to Rsymphony and an interface to Lpsolve.
>
> Performance-wise Lpsolve is very poor, but I know that it has a community of 
> devoted followers. I've not seen benchmark results for Symphony, but I know 
> that Cbc is the preferred COIN-OR MIP solver when it comes to general 
> performance.  And, as I observed, the performance of HiGHS is way better than 
> Cbc.
>
> Are people in the R community tearing their hair out over the performance of 
> software requiring the solution of LPs or MIPs?
>
> Would a significantly better LP/MIP solver be valuable to the R community?
>
> Thanks,
>
> Julian
> --
> Dr. J. A. Julian Hall, Reader, School of Mathematics,
> University of Edinburgh, James Clerk Maxwell Building,
> Peter Guthrie Tait Road, EDINBURGH, EH9 3FD, UK.
> Room: 5418 Phone: [+44](131) 650 5075 Email: 
> j.a.j.h...@ed.ac.uk
> Web: https://www.maths.ed.ac.uk/school-of-mathematics/people/a-z?person=47
> [HiGHS]
>
> My working hours may not be your working hours. Do not feel pressure to reply 
> to this email outside your working hours.
> The University of Edinburgh is a charitable body, registered in Scotland, 
> with registration number SC005336. Is e buidheann carthannais a th’ ann an 
> Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

Hello, Julian.

I cannot speak for the R community, but as someone who needs
optimization on a regular basis, this sounds intriguing. The fact that
HiGHS appears to be FLOSS, and thus usable as-is in the corporate
setting, appeals to those of us who use R in industry. Would you have
any statistics on how the solvers in HiGHS compare with similar ones
currently available in R, specifically the following in NLOPT [1]
(which is called through nloptr): SLSQP (gradient-based) and COBYLA
(gradient-free) both of which support equality and inequality
constraints, and MMA/CCSA (gradient based) which supports inequality
constraints? As for integer or mixed integer programming, I believe
that there is a lot of room for improvement in R. Personally, I've
resorted to using DEOptim with the "fnMap" entry calling a round
function similar to [2]. So speaking for myself, giving richer options
for optimization is a good thing, especially if the installation
procedure can be simplified!

Thank you,

Avi

[1] https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/
[2] 
https://stackoverflow.com/questions/42197353/how-to-set-integer-constraint-using-fnmap-in-deoptim-r

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Improved LP/MIP solver

[Avraham Adler]

On Sun, Dec 12, 2021 at 4:24 PM Avraham Adler  wrote:
>
> On Sun, Dec 12, 2021 at 3:44 PM Julian Hall  wrote:
> >
> > Dear All,
> >
> > I am leading the development of HiGHS, which is now the top performing open 
> > source linear optimization software on the industry standard benchmarks. In 
> > particular, our MIP solver out-performs SCIP, and is way ahead of the 
> > COIN-OR solver Cbc.
> >
> > HiGHS solves LPs via simplex or interior point, MIPs via branch-and-cut, 
> > and QPs via an active set method.
> >
> > We were wondering what interest there would be in developing an R interface 
> > to HiGHS. I'm not an R user, but have done a bit of searching and see 
> > references to Rsymphony and an interface to Lpsolve.
> >
> > Performance-wise Lpsolve is very poor, but I know that it has a community 
> > of devoted followers. I've not seen benchmark results for Symphony, but I 
> > know that Cbc is the preferred COIN-OR MIP solver when it comes to general 
> > performance.  And, as I observed, the performance of HiGHS is way better 
> > than Cbc.
> >
> > Are people in the R community tearing their hair out over the performance 
> > of software requiring the solution of LPs or MIPs?
> >
> > Would a significantly better LP/MIP solver be valuable to the R community?
> >
> > Thanks,
> >
> > Julian
> > --
> > Dr. J. A. Julian Hall, Reader, School of Mathematics,
> > University of Edinburgh, James Clerk Maxwell Building,
> > Peter Guthrie Tait Road, EDINBURGH, EH9 3FD, UK.
> > Room: 5418 Phone: [+44](131) 650 5075 Email: 
> > j.a.j.h...@ed.ac.uk<mailto:j.a.j.h...@ed.ac.uk>
> > Web: https://www.maths.ed.ac.uk/school-of-mathematics/people/a-z?person=47
> > [HiGHS]<http://www.highs.dev>
> >
> > My working hours may not be your working hours. Do not feel pressure to 
> > reply to this email outside your working hours.
> > The University of Edinburgh is a charitable body, registered in Scotland, 
> > with registration number SC005336. Is e buidheann carthannais a th’ ann an 
> > Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
>
> Hello, Julian.
>
> I cannot speak for the R community, but as someone who needs
> optimization on a regular basis, this sounds intriguing. The fact that
> HiGHS appears to be FLOSS, and thus usable as-is in the corporate
> setting, appeals to those of us who use R in industry. Would you have
> any statistics on how the solvers in HiGHS compare with similar ones
> currently available in R, specifically the following in NLOPT [1]
> (which is called through nloptr): SLSQP (gradient-based) and COBYLA
> (gradient-free) both of which support equality and inequality
> constraints, and MMA/CCSA (gradient based) which supports inequality
> constraints? As for integer or mixed integer programming, I believe
> that there is a lot of room for improvement in R. Personally, I've
> resorted to using DEOptim with the "fnMap" entry calling a round
> function similar to [2]. So speaking for myself, giving richer options
> for optimization is a good thing, especially if the installation
> procedure can be simplified!
>
> Thank you,
>
> Avi
>
> [1] https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/
> [2] 
> https://stackoverflow.com/questions/42197353/how-to-set-integer-constraint-using-fnmap-in-deoptim-r

Also, to be good R-citizens, this thread should probably be moved to
R-package-devel [1].

Thanks,

Avi

[1] https://stat.ethz.ch/mailman/listinfo/r-package-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Appropriate mailing list for " Improved LP/MIP solver "

[Avraham Adler]

I defer completely and totally to Martin!

Apologies,

Avi

On Mon, Dec 13, 2021 at 10:59 AM Martin Maechler 
wrote:

> >>>>> Avraham Adler
> >>>>> on Sun, 12 Dec 2021 16:27:02 + writes:
>
>  []
>
> Thank you, Julian and Avi,  on the topic of asking and replying
> if there's interest on getting improved LP / MIP  (and QP, I think was
> mentioned too!) solver interfaces for R.
>
> However, Avi writes
>
> > Also, to be good R-citizens, this thread should probably be moved to
> > R-package-devel [1].
>
> > Thanks,
> > Avi
>
> and I'm of an entirely different opinion.
>
> R-package-devel  has been created, aons after R-devel,  to
> *help* R package developers to get their packaging problems
> solved, notably to get advice in making their packages ready for
> CRAN.
>
> Julian's question was really addressing the whole R developer
> community asking if some functionality was desirable to be added
> to the R-package space.
>
> For me one *the* appropriate topics for this R-devel mailing
> list.
>
> Best,
> Martin
>
> ---
> Martin Maechler
> ETH Zurich  and  R Core
> (and original creator of R-help, R-devel, .. lists)
>
> > [1] https://stat.ethz.ch/mailman/listinfo/r-package-devel
>
>
> --
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Parser bug? A comma too much.

[Avraham Adler]

That may actually be the case for EVERY default plot, if I understand
correctly. The default S# plot is defined as

## Default S3 method:
plot(x, y = NULL, type = "p",  xlim = NULL, ylim = NULL,
 log = "", main = NULL, sub = NULL, xlab = NULL, ylab = NULL,
 ann = par("ann"), axes = TRUE, frame.plot = axes,
 panel.first = NULL, panel.last = NULL, asp = NA,
 xgap.axis = NA, ygap.axis = NA,
 ...)

By putting in the comma, unless I am mistaken, you are effectively
saying the second element is NULL, which is how it's naturally
defined.

Thanks,

Avi

On Fri, Sep 16, 2022 at 2:33 PM Rui Barradas  wrote:
>
> Hello,
>
> Why doesn't the comma throw an error?
>
>
> plot(1:5,)# works, no complaints
>
>
> Shouldn't this be considered a bug? I would expect the parser to at
> least give a warning.
>
>
> sessionInfo()
> R version 4.2.1 (2022-06-23 ucrt)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows 10 x64 (build 22000)
>
> Matrix products: default
>
> locale:
> [1] LC_COLLATE=Portuguese_Portugal.utf8
> LC_CTYPE=Portuguese_Portugal.utf8
> [3] LC_MONETARY=Portuguese_Portugal.utf8 LC_NUMERIC=C
>
> [5] LC_TIME=Portuguese_Portugal.utf8
>
> attached base packages:
> [1] stats graphics  grDevices utils datasets  methods   base
>
> loaded via a namespace (and not attached):
> [1] compiler_4.2.1
>
>
> Rui Barradas
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Linking to Intel's MKL on Windows

[Avraham Adler]

Also, you can build Rblas against OpenBLAS even on Windows which will go far in 
speeding up matrix calculations. 

Thanks,

Avi

Sent from my iPhone

> On Oct 1, 2022, at 12:49 PM, Ben Bolker  wrote:
> 
>    Maybe you can find out more about Microsoft's development/release process 
> for MRO and why they're still on 4.0.2 (from June 2020)?  I followed the 
> "user forum" link on their web page, but it appears to be a generic Windows 
> forum ...
> 
> https://social.msdn.microsoft.com/Forums/en-US/home?forum%20=ropen
> 
>   I might tweet at @revodavid (David Smith) to see if there's any more 
> information available about the MRO release schedule ...
> 
>  good luck,
>   Ben Bolker
> 
> 
> 
> 
>> On 2022-10-01 12:00 p.m., Viechtbauer, Wolfgang (NP) wrote:
>> Hi Christine,
>> MKL is a closed-source commercial product (yes, one can get it for free, but 
>> it is not libre/open-source software).
>> Best,
>> Wolfgang
>>> -Original Message-
>>> From: R-devel [mailto:r-devel-boun...@r-project.org] On Behalf Of Christine
>>> Stawitz - NOAA Federal via R-devel
>>> Sent: Friday, 30 September, 2022 18:46
>>> To: r-devel@r-project.org
>>> Subject: [Rd] Linking to Intel's MKL on Windows
>>> 
>>> Hi,
>>> 
>>> Recently I became aware that Microsoft R Open provides accelerated matrix
>>> algebra computations through Intel's Math Kernel Libraries. However, the
>>> version of R shipped with the Microsoft R Open is too out of date to be
>>> able to use concurrently with other dependencies while developing our
>>> package. This thread suggests a way to get the updated matrix libraries
>>> with a more recent version of R, however it is unlikely to be approved by
>>> our IT admin since those of us in government agencies aren't typically
>>> given admin privileges: Linking Intel's Math Kernel Library (MKL) to R on
>>> Windows - Stack Overflow
>>> >> mkl-to-r-on-windows>
>>> 
>>> Is there a reason why CRAN doesn't provide a version of R with the updated
>>> libraries such that developers don't have to recompile R or copy .dlls
>>> around as described above? It would help those of us running software with
>>> slow-running matrix calculations in R.
>>> 
>>> Thanks,
>>> Christine
>>> 
>>> --
>>> Christine C. Stawitz, PhD. (pronouns: she/her)
>>> 
>>> National Stock Assessment Program Modeling Team
>>> 
>>> NOAA Fisheries Office of Science and Technology |  U.S. Department of
>>> Commerce
>>> 
>>> Mobile: 206-617-2060
>>> 
>>> www.fisheries.noaa.gov
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> -- 
> Dr. Benjamin Bolker
> Professor, Mathematics & Statistics and Biology, McMaster University
> Director, School of Computational Science and Engineering
> (Acting) Graduate chair, Mathematics & Statistics
> > E-mail is sent at my convenience; I don't expect replies outside of working 
> > hours.
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] R-devel 83314 fails datetime3 on Windows

[Avraham Adler]

Hello.

Building 83314 on R using R-tools I get the following failure which
relates to lines 463--470 in the datetime3.R test:

```
> ifi3 <- is.finite(dctm3)
> stopifnot(exprs = {
+ all.equal(dD, dDc, tolerance = 1e-4)
+ (dDm3 - dDcm3)[ifi3] %in% 0:1
+   (dD - dDc  )[ifi]  %in% 0:1
+   (nD - nDc  )[ifi]  %in% 0:1
+ is.na((dD   - dDc  )[!ifi])
+ is.na((dDm3 - dDcm3)[!ifi3])
+ })
Error: (dDm3 - dDcm3)[ifi3] %in% 0:1 are not all TRUE
Execution halted
```

In particular, please note that the first entries in the following two
vectors differ by one day:

```
> dDm3
 [1] "2016-12-06" NA   NA   "2016-12-06" NA
"2016-04-06" NA   NA   "2016-04-07"
[10] "2016-12-06" NA   "-Inf"   NA
> dDcm3
 [1] "2016-12-07" NA   NA   "2016-12-06" NA
"2016-04-06" NA   NA   "2016-04-07"
[10] "2016-12-06" NA   "-Inf"   NA
```
Also, in the sessionInfo below, I wonder why my computer returns a
time zone of Australia/Melbourne when I am based in US/New York

Session Info:

R Under development (unstable) (2022-11-08 r83314 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United
States.utf8LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C   LC_TIME=English_United
States.utf8

time zone: Australia/Melbourne
tzcode source: internal

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_4.3.0

Note that Rblas is based on OpenBLAS 0.3.21

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R-4.2.3 build from source on Windows (w Rtools42) - lto1.exe error

[Avraham Adler]

Hi. 

Are you sure you have the most up-to-date version of the libraries? It should 
be version 5550. Tomas updates the entire toolchain semi-regularly. If you call 
“ cat /x86_64-w64-mingw32.static.posix/.version” and it’s not 5550, try 
updating the libraries as per 
https://cran.r-project.org/bin/windows/base/howto-R-devel.html.

Hope that helps,

Avi

Sent from my iPhone

> On May 10, 2023, at 9:07 PM, Gmail  wrote:
> 
> Windows 11 PRO Version  10.0.22621 Build 22621
> Processor: Intel(R) Core(TM) i7-1065G7 "Icelake-client"
> ```
> ​svn info
> Path: .
> Working Copy Root Path: /d/R_DEV/R-4/R42/R-4-2-branch
> URL: https://svn.r-project.org/R/branches/R-4-2-branch
> Relative URL: ^/branches/R-4-2-branch
> Repository Root: https://svn.r-project.org/R
> Repository UUID: 00db46b3-68df-0310-9c12-caf00c1e9a41
> Revision: 84417
> Node Kind: directory
> Schedule: normal
> Last Changed Author: kalibera
> Last Changed Rev: 84249
> Last Changed Date: 2023-04-13 07:12:24 + (Thu, 13 Apr 2023)
> ```
> 
> Only adaptation done in MkRules.local was adding: `EOPTS = -march=native`  - 
> that's why I included the cpu-type info above;
> running make all recommended​ fails at/with:
> ```
> gcc -shared -s -static-libgcc -o utils.dll tmp.def init.o io.o size.o sock.o 
> stubs.o utils.o hashtab.o windows/dataentry.o windows/dialogs.o 
> windows/registry.o windows/util.o windows/widgets.o 
> ../../../gnuwin32/dllversion.o -lRgraphapp -lversion 
> -L/x86_64-w64-mingw32.static.posix/lib/x64 -llzma 
> -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib/x64 
> -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib -L../../../../bin/x64 -lR
> 
> lto1.exe: fatal error: bytecode stream in file 'windows/dataentry.o' 
> generated with LTO version 9.3 instead of the expected 9.4
> compilation terminated.
> 
> lto-wrapper.exe: fatal error: 
> C:\rtools42\x86_64-w64-mingw32.static.posix\bin\gcc.exe returned 1 exit status
> compilation terminated.
> C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: error: lto-wrapper 
> failed
> collect2.exe: error: ld returned 1 exit status
> cp: cannot stat 'utils.dll': No such file or directory
> make[4]: *** [Makefile.win:36: shlib] Error 1
> make[3]: *** [../../../share/make/basepkg.mk:145: mksrc-win2] Error 1
> make[2]: *** [Makefile.win:24: all] Error 2
> make[1]: *** [Makefile.win:34: R] Error 1
> make: *** [Makefile:18: all] Error 2
> ```
> Any hints/ideas on how to fix this? I guess I could 
> gcc -c -flto ... windows/dataentry.c -o windows/dataentry.o 
> with the exact path of that fiile ... 
> and it hopefully will fix that but I guess it would make sense to add a 
> Revision to update that LTO version mismatch there, and I don't know yet if 
> this is the only one?
> 
> Greetings,
> W
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Segmentation fault early in compilation of revision 85514

[Avraham Adler]

When trying to compile R on Windows 10 64-bit using LTO as I always do, I
encountered a segmentation fault early in the compilation. I am uncertain
as to what it means (please see below for error extract). I am using the
most recent version of Rtools43 (5863) and updated its libraries prior to
starting the build. My EOPTS is " -march=native -pipe -mno-rtm" and my
LTO/LTO_OPT/LTO_FC/LTO_FC_OPT is "-flto=1 -fuse-linker-plugin".

Any explanation or suggestions would be appreciated.

Thank you,

Avi

```
installing zoneinfo
making console.d from console.c
making dynload.d from dynload.c
making embeddedR.d from embeddedR.c
making preferences.d from preferences.c
making psignal.d from psignal.c
making rui.d from rui.c
making system.d from system.c
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
console.c -o console.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
dynload.c -o dynload.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
editor.c -o editor.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
embeddedR.c -o embeddedR.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD
-I../library/grDevices/src -O3 -Wall -pedantic -march=native -pipe -mno-rtm
-flto=1 -fuse-linker-plugin   -c extra.c -o extra.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
opt.c -o opt.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
pager.c -o pager.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
preferences.c -o preferences.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
psignal.c -o psignal.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
rhome.c -o rhome.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
rt_complete.c -o rt_complete.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
rui.c -o rui.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
run.c -o run.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD  -O3 -Wall
-pedantic -march=native -pipe -mno-rtm -flto=1 -fuse-linker-plugin   -c
sys-win32.c -o sys-win32.o
gcc  -I../include -I. -I../extra -DHAVE_CONFIG_H -DR_DLL_BUILD
-DR_ARCH='"x64"' -O3 -Wall -pedantic -march=native -pipe -mno-rtm -flto=1
-fuse-linker-plugin   -c system.c -o system.o
windres   -I../include -i dllversion.rc -o dllversion.o
C:\rtools43\x86_64-w64-mingw32.static.posix\bin/nm.exe: mlutils.o: no
symbols
C:\rtools43\x86_64-w64-mingw32.static.posix\bin/nm.exe: log1p.o: no symbols
C:\rtools43\x86_64-w64-mingw32.static.posix\bin/nm.exe: printf.o: no symbols
C:\rtools43\x86_64-w64-mingw32.static.posix\bin/nm.exe: osdep.o: no symbols
gcc  -shared -O3 -Wall -pedantic -march=native -pipe -mno-rtm -flto=1
-fuse-linker-plugin -s -mwindows -o R.dll R.def console.o dynload.o
editor.o embeddedR.o extra.o opt.o pager.o preferences.o psignal.o rhome.o
rt_complete.o rui.o run.o shext.o sys-win32.o system.o dos_wglob.o
dllversion.o ../main/libmain.a ../appl/libappl.a ../nmath/libnmath.a
getline/gl.a ../extra/xdr/libxdr.a ../extra/intl/libintl.a
../extra/trio/libtrio.a ../extra/tzone/libtz.a ../extra/tre/libtre.a
-fopenmp -L. -lgfortran -lquadmath -lRblas -L../../bin/x64 -lRgraphapp
-lRiconv -lcomctl32 -lole32 -luuid -lwinmm -lversion
-L"/x86_64-w64-mingw32.static.posix"/lib/x64 -lpcre2-8 -lz -lbz2 -llzma
-L""/lib/x64 -lsicuin -lsicuuc
/x86_64-w64-mingw32.static.posix/lib/sicudt.a -lstdc++
rui.h:37:18: warning: type of 'RConsole' does not match original
declaration [-Wlto-type-mismatch]
   37 | LibExtern window RConsole;
  |  ^
rui.c:51:9: note: type 'struct objinfo' should match type 'struct gui_obj'
   51 | console RConsole = NULL;
  | ^
rui.c:51:9: note: 'RConsole' was previously declared here
rui.c:51:9: note: code may be misoptimized unless '-fno-strict-aliasing' is
used
make[5]: [C:\rtools43\tmp\ccUw2zA8.mk:81:
C:\rtools43\tmp\ccKMP0O7.ltrans26.ltrans.o] Error 1 (ignore
d)
make[5]: [C:\rtools43\tmp\ccUw2zA

Re: [Rd] Segmentation fault early in compilation of revision 85514

[Avraham Adler]

On Mon, Nov 13, 2023 at 1:13 AM Dirk Eddelbuettel  wrote:
>
>
> Avi,
>
> Might be toolchain-dependent, might be options-dependent--it built fine here.
> Easier for you to vary option two so maybe try that?
>
> Dirk

Thank you, Dirk.

I think it was more a PEBCAK issue. When I deleted the entire trunk
folder and started the process from scratch, it compiled properly as
per usual. Although this was revision 85520, so perhaps something
changed in the interim.

Regardless, the issue resolved itself.

Thanks,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Advice debugging M1Mac check errors

[Avraham Adler]

I think I had that problem with the minimaxApprox package as well.

Avi

Sent from my iPhone

> On Feb 4, 2024, at 4:47 PM, J C Nash  wrote:
> 
> Slightly tangential: I had some woes with some vignettes in my
> optimx and nlsr packages (actually in examples comparing to OTHER
> packages) because the M? processors don't have 80 bit registers of
> the old IEEE 754 arithmetic, so some existing "tolerances" are too
> small when looking to see if is small enough to "converge", and one
> gets "did not converge" type errors. There are workarounds,
> but the discussion is beyond this post. However, worth awareness that
> the code may be mostly correct except for appropriate tests of
> smallness for these processors.
> 
> JN
> 
> 
> 
> 
>> On 2024-02-04 11:51, Dirk Eddelbuettel wrote:
>> On 4 February 2024 at 20:41, Holger Hoefling wrote:
>> | I wanted to ask if people have good advice on how to debug M1Mac package
>> | check errors when you don´t have a Mac? Is a cloud machine the best option
>> | or is there something else?
>> a) Use the 'mac builder' CRAN offers:
>>https://mac.r-project.org/macbuilder/submit.html
>> b) Use the newly added M1 runners at GitHub Actions,
>>
>> https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/
>> Option a) is pretty good as the machine is set up for CRAN and builds
>> fast. Option b) gives you more control should you need it.
>> Dirk
>> 
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Change between 86152 and 86534 - probably 86265 - that looks for zspmv in BLAS and not LAPACK causes R with OpenBLAS to fail

[Avraham Adler]

Executive summary:

I believe revision 86265 makes it more difficult to build R with
OpenBLAS on Windows as now the entire LAPACK needs to be built to
obtain zspmv. Is there anything that can be done to allow the former
behavior to be used, something in Mkrules.local perhaps?

Detailed Explanation:

I have been building R with OpenBLAS for Windows 64 for over a decade
by patching /src/extra/blas/Makevars.win as follows:

--- /c/r/trunk/src/extra/blas/Makefile.win2024-01-24
18:34:42.755255900 +
+++ /c/r/Makefile.win2024-01-24 18:39:39.716458000 +
@@ -12,7 +12,7 @@
 ../../../$(BINDIR)/Rblas.dll: blas00.o ../../gnuwin32/dllversion.o
 @$(ECHO)  Building $@ 
 $(DLL) -s -shared $(DLLFLAGS) -o $@ $^ Rblas.def \
-   -L../../../$(IMPDIR) -lR  -L"$(ATLAS_PATH)" -lf77blas -latlas
+   -L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -fopenmp -lopenblas
 else
 ../../../$(BINDIR)/Rblas.dll: blas.o blas2.o cmplxblas.o cmplxblas2.o
../../gnuwin32/dllversion.o
 @$(ECHO)  Building $@ 

and then passing USE_ATLAS = YES and ATLAS_PATH = C:/R/OPB/whatever in
Mkrules.local

When I compile OpenBLAS, I have always done so with NO_CBLAS,
NO_LAPACK, and NO_SHARED, as the Windows toolchain does not need
CBLAS, a shared library, or allow for a separate LAPACK and this has
worked, for the most part, since roughly 2013.
When building the recent R-devel (a revision slightly before 86534),
the compilation stopped when building Rblas with the following error:

 Building ../../../bin/x64/Rblas.dll 
gcc  -s -shared  -o ../../../bin/x64/Rblas.dll blas00.o
../../gnuwin32/dllversion.o Rblas.def \
   -L../../../bin/x64 -lR -L"C:/R/OPB/OpenBLAS-develop-f0560f9"
-fopenmp -lopenblas
C:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export
zspmv_: symbol not defined
collect2.exe: error: ld returned 1 exit status
make[4]: *** [Makefile.win:14: ../../../bin/x64/Rblas.dll] Error 1
make[3]: *** [Makefile:227: Rblas] Error 2
make[2]: *** [Makefile:116: rbuild] Error 2
make[1]: *** [Makefile:17: all] Error 2
make: *** [Makefile:392: distribution] Error 2

I reached out to OpenBLAS (there were other issues with OPB 0.3.27)
and one of the maintainers responded:

> "for historical reasons, ZSPMV is in LAPACK although conceptionally [sic] it 
> belongs in BLAS. As you specified NO_LAPACK=1, this function gets omitted 
> since 0.3.20 to allow combining the BLAS-only build with an external LAPACK." 
> [1]

He later followed up with "0.3.26 built with NO_LAPACK=1 will likewise
omit the zspmv symbol (as directed)." [2]

I last successfully compiled v86152 with OpenBLAS 0.3.26 on
2024-03-19. When I compiled 86534 tonight with OPB 0.3.27, I got the
error above. I then tried with OPB 0.3.26—which worked for 86152—and
still got the zspmv error.

I am guessing this relates to revision 86265 "amending r85873: zspmv
is BLAS, not Lapack…" [3].

I built OpenBLAS AND its LAPACK, which takes MUCH MUCH longer, and
tried building R. This time, the build succeeded and passes make
check-devel.

Is there any way to allow the former functionality if the build
recognizes the use of OpenBLAS? Is the only option to compile
OpenBLAS's LAPACK too?

Thank you,

Avi


[1] https://github.com/OpenMathLib/OpenBLAS/issues/4684#issuecomment-2101123154
[2] https://github.com/OpenMathLib/OpenBLAS/issues/4684#issuecomment-2101213474
[3] 
https://github.com/r-devel/r-svn/commit/c9f3aba39aa89821d294f4a524331a21e6904aec

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Failing lm-tests due to extra 0 in scientific notation?

[Avraham Adler]

Hello.

I've compiled R on Windows many times, and this is the first time I've
seen this error. While running make check-all (and using
testInstalledBasic("both")), the lm-tests routines fail, and, as far
as I can tell, the diff is failing because in one file, answers are
coming back like this "3.11e-004" while in the save file they are
"3.11e-04". Every value is the same, outside the extra 0 in the
scientific notation. I've never seen R put two 0s in a row like that
before, and I cannot think of why that would happen. Is there a way to
change that so that it passes the tests?

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Failing lm-tests due to extra 0 in scientific notation?

[Avraham Adler]

Please let me clarify. When I said "is there a way to change that," I
meant, does anyone know why R would respond that way, and does anyone have
any suggestions as to what I can do or what I should investigate to get my
compilation to conform. I did *not* mean, "can we change the reference." I
apologize for any unintentional implications.

Avi

On Wed, Jan 7, 2015 at 1:24 AM, Avraham Adler 
wrote:

> Hello.
>
> I've compiled R on Windows many times, and this is the first time I've
> seen this error. While running make check-all (and using
> testInstalledBasic("both")), the lm-tests routines fail, and, as far
> as I can tell, the diff is failing because in one file, answers are
> coming back like this "3.11e-004" while in the save file they are
> "3.11e-04". Every value is the same, outside the extra 0 in the
> scientific notation. I've never seen R put two 0s in a row like that
> before, and I cannot think of why that would happen. Is there a way to
> change that so that it passes the tests?
>
> Thank you,
>
> Avi
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] New version of Rtools for Windows

[Avraham Adler]

Very timely, as this is how I got into the problem I posted about
earlier; maybe some of the problems I ran into will mean more to the
you and the experts on this thread, Dr. Murdoch.For reference, I run
Windows 7 64bit, and I am trying to build a 64 bit version of R-3.1.2.

As we discussed offline, Dr. Murdoch, I've been trying to build R
using more recent tools than GCC4.6.3 prerelease. Ruben Von Boxen
(rubenvb) told me he is no longer developing his own builds of GCC,
but is focusing on MSYS2 and the mingw64 personal builds. So, similar
to what Jeroen said, I first installed MSYS2, whose initial
installation on windows is not so simple[1]. After the initial
install, the following packages need to be manually installed: make,
tar, zip, unzip, zlib, and rsync. I also installed base-devel, which
is way more than necessary, but there may be packages in there which
are necessary.

I originally installed the most up-to-date version of GCC (4.9.2)[2],
and I did pick the -seh version, as since I install (almost) all
packages from source (the one exception being nloptr for now), the
exception handling should be consistent and it is supposed to up to
~15% faster[3].

The initial build crashed with the following error:

gcc -std=gnu99 -m64 -I../../include -I. -DHAVE_CONFIG_H  -O3 -Wall
-pedantic -mtune=core2   -c xmalloc.c -o xmalloc.o
ar crs libtre.a regcomp.o regerror.o regexec.o tre-ast.o tre-compile.o
tre-match -approx.o tre-match-backtrack.o tre-match-parallel.o
tre-mem.o tre-parse.o tre-stack.o xmalloc.o
gcc -std=gnu99 -m64   -O3 -Wall -pedantic -mtune=core2   -c compat.c -o compat.o
compat.c:65:5: error: redefinition of 'snprintf'
 int snprintf(char *buffer, size_t max, const char *format, ...)
 ^
In file included from compat.c:3:0:
F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:553:5: note: previous
definition of 'snprintf' was here
 int snprintf (char * __restrict__ __stream, size_t __n, const char *
__restrict__ __format, ...)
 ^
compat.c:75:5: error: redefinition of 'vsnprintf'
 int vsnprintf(char *buffer, size_t bufferSize, const char *format,
va_list args)
 ^
In file included from compat.c:3:0:
F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:543:7: note: previous
definition of 'vsnprintf' was here
   int vsnprintf (char * __restrict__ __stream, size_t __n, const char
* __restrict__ __format, va_list __local_argv)
   ^
../../gnuwin32/MkRules:218: recipe for target 'compat.o' failed
make[4]: *** [compat.o] Error 1
Makefile:120: recipe for target 'rlibs' failed
make[3]: *** [rlibs] Error 1
Makefile:179: recipe for target '../../bin/x64/R.dll' failed
make[2]: *** [../../bin/x64/R.dll] Error 2
Makefile:104: recipe for target 'rbuild' failed
make[1]: *** [rbuild] Error 2
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 2

After doing some checking (for example see [4]), I asked Duncan about
the problem, and he suggested moving the #ifndef _W64 in compat.c up
above the offending lines (65-75). That did not work, so, I figured
(it seems mistakenly from the other thread) that if those functions
are included from stdio already, I can just delete them from compat.c.
The specific lines are:

int snprintf(char *buffer, size_t max, const char *format, ...)
{
int res;
va_list(ap);
va_start(ap, format);
res = trio_vsnprintf(buffer, max, format, ap);
va_end(ap);
return res;
}

int vsnprintf(char *buffer, size_t bufferSize, const char *format, va_list args)
{
return trio_vsnprintf(buffer, bufferSize, format, args);
}

Continuing the build using 4.9.2 crashed again at the following point:

gcc -std=gnu99 -m64 -I../include -I. -I../extra -DHAVE_CONFIG_H
-DR_DLL_BUILD  -O3 -Wall -pedantic -mtune=core2   -c malloc.c -o
malloc.o
windres -F pe-x86-64  -I../include -i dllversion.rc -o dllversion.o
gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
dynload.o editor.o embeddedR.o extra.o opt.o pager.o preferences.o
psignal.o rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o
system.o dos_wglob.o malloc.o ../main/libmain.a ../appl/libappl.a
../nmath/libnmath.a getline/gl.a ../extra/xdr/libxdr.a
../extra/pcre/libpcre.a ../extra/bzip2/libbz2.a
../extra/intl/libintl.a ../extra/trio/libtrio.a ../extra/tzone/libtz.a
../extra/tre/libtre.a ../extra/xz/liblzma.a dllversion.o -fopenmp -L.
-lgfortran -lRblas -L../../bin/x64 -lRzlib -lRgraphapp -lRiconv
-lcomctl32 -lversion
collect2.exe: error: ld returned 5 exit status
Makefile:150: recipe for target 'R.dll' failed
make[3]: *** [R.dll] Error 1
Makefile:179: recipe for target '../../bin/x64/R.dll' failed
make[2]: *** [../../bin/x64/R.dll] Error 2
Makefile:104: recipe for target 'rbuild' failed
make[1]: *** [rbuild] Error 2
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 2

As all those files existed in their correct places, the only reason I
could think of that this would fail here is that GCC version 4.9 did
make some changes to enhance link-time optimization [5], and probably
something isn't com

Re: [Rd] New version of Rtools for Windows

[Avraham Adler]

On Thu, Jan 8, 2015 at 10:48 AM, Hin-Tak Leung
 wrote:
>
> The r.dll crash is easy - you need to be using gcc-ar for ar, and gcc-ranlib 
> for ranlib. I also posted a patch to fix the check failure for stack probing, 
> as lto optimizes away the stack probing code, as it should.
>
> yes, lto build's speed gain is very impressive.
>
>


I apologize for my ignorance, but how would I do that? I tried by
changing the following in src/gnuwin32/MkRules.local:

# prefix for 64-bit: path or x86_64-w64-mingw32-
BINPREF64 = x86_64-w64-mingw32-gcc-

I added the gcc- as the suffix there, but I guess that is insufficient
as I still get the following error using 4.9.2:

windres -F pe-x86-64  -I../include -i dllversion.rc -o dllversion.o
gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
dynload.o editor.o embeddedR.o extra.o opt.o pager.o preferences.o
psignal.o rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o
system.o dos_wglob.o malloc.o ../main/libmain.a ../appl/libappl.a
../nmath/libnmath.a getline/gl.a ../extra/xdr/libxdr.a
../extra/pcre/libpcre.a ../extra/bzip2/libbz2.a
../extra/intl/libintl.a ../extra/trio/libtrio.a ../extra/tzone/libtz.a
../extra/tre/libtre.a ../extra/xz/liblzma.a dllversion.o -fopenmp -L.
-lgfortran -lRblas -L../../bin/x64 -lRzlib -lRgraphapp -lRiconv
-lcomctl32 -lversion
collect2.exe: error: ld returned 5 exit status
Makefile:150: recipe for target 'R.dll' failed
make[3]: *** [R.dll] Error 1
Makefile:179: recipe for target '../../bin/x64/R.dll' failed
make[2]: *** [../../bin/x64/R.dll] Error 2
Makefile:104: recipe for target 'rbuild' failed
make[1]: *** [rbuild] Error 2
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 2

I still had to delete those lines in compat.c, so this build, were it
to have completed, is still subject to the non-conformance of
scientfic notation printing that was discussed earlier.

Hin-tak, any suggestions for this error (and the compat.c for that
matter) that you, or any reader of this list, may have would be
greatly appreciated.

Thank you!

Avi



> --
> On Thu, Jan 8, 2015 2:01 PM GMT Henric Winell wrote:
>
>>On 2015-01-08 14:18, Avraham Adler wrote:
>>
>>> Very timely, as this is how I got into the problem I posted about
>>> earlier; maybe some of the problems I ran into will mean more to the
>>> you and the experts on this thread, Dr. Murdoch.For reference, I run
>>> Windows 7 64bit, and I am trying to build a 64 bit version of R-3.1.2.
>>>
>>> As we discussed offline, Dr. Murdoch, I've been trying to build R
>>> using more recent tools than GCC4.6.3 prerelease. Ruben Von Boxen
>>> (rubenvb) told me he is no longer developing his own builds of GCC,
>>> but is focusing on MSYS2 and the mingw64 personal builds. So, similar
>>> to what Jeroen said, I first installed MSYS2, whose initial
>>> installation on windows is not so simple[1]. After the initial
>>> install, the following packages need to be manually installed: make,
>>> tar, zip, unzip, zlib, and rsync. I also installed base-devel, which
>>> is way more than necessary, but there may be packages in there which
>>> are necessary.
>>>
>>> I originally installed the most up-to-date version of GCC (4.9.2)[2],
>>> and I did pick the -seh version, as since I install (almost) all
>>> packages from source (the one exception being nloptr for now), the
>>> exception handling should be consistent and it is supposed to up to
>>> ~15% faster[3].
>>>
>>> The initial build crashed with the following error:
>>>
>>> gcc -std=gnu99 -m64 -I../../include -I. -DHAVE_CONFIG_H  -O3 -Wall
>>> -pedantic -mtune=core2   -c xmalloc.c -o xmalloc.o
>>> ar crs libtre.a regcomp.o regerror.o regexec.o tre-ast.o tre-compile.o
>>> tre-match -approx.o tre-match-backtrack.o tre-match-parallel.o
>>> tre-mem.o tre-parse.o tre-stack.o xmalloc.o
>>> gcc -std=gnu99 -m64   -O3 -Wall -pedantic -mtune=core2   -c compat.c -o 
>>> compat.o
>>> compat.c:65:5: error: redefinition of 'snprintf'
>>>   int snprintf(char *buffer, size_t max, const char *format, ...)
>>>   ^
>>> In file included from compat.c:3:0:
>>> F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:553:5: note: previous
>>> definition of 'snprintf' was here
>>>   int snprintf (char * __restrict__ __stream, size_t __n, const char *
>>> __restrict__ __format, ...)
>>>   ^
>>> compat.c:75:5: error: redefinition of 'vsnprintf'
>>>   int vsnprintf(char *buffer, size_t bufferSize, const char *format,
>>> va_list args)
>>

Re: [Rd] New version of Rtools for Windows

[Avraham Adler]

Regarding the redefinition error, I've asked on StackOverflow for
advice [1], but I have noticed the following; perhaps someone here can
understand what changed between the stdio.h of 4.6.3 and the stdio.h
of 4.8.4. In GCC 4.8.4, the section of stdio.h which is referenced in
the errors is the following:

#if !defined (__USE_MINGW_ANSI_STDIO) || __USE_MINGW_ANSI_STDIO == 0
/* this is here to deal with software defining
 * vsnprintf as _vsnprintf, eg. libxml2.  */
#pragma push_macro("snprintf")
#pragma push_macro("vsnprintf")
# undef snprintf
# undef vsnprintf
  int __cdecl __ms_vsnprintf(char * __restrict__ d,size_t n,const char
* __restrict__ format,va_list arg)
__MINGW_ATTRIB_DEPRECATED_MSVC2005 __MINGW_ATTRIB_DEPRECATED_SEC_WARN;

  __mingw_ovr
  __MINGW_ATTRIB_NONNULL(3)
  int vsnprintf (char * __restrict__ __stream, size_t __n, const char
* __restrict__ __format, va_list __local_argv)
  {
return __ms_vsnprintf (__stream, __n, __format, __local_argv);
  }

  int __cdecl __ms_snprintf(char * __restrict__ s, size_t n, const
char * __restrict__  format, ...);

#ifndef __NO_ISOCEXT
__mingw_ovr
__MINGW_ATTRIB_NONNULL(3)
int snprintf (char * __restrict__ __stream, size_t __n, const char *
__restrict__ __format, ...)
{
  register int __retval;
  __builtin_va_list __local_argv; __builtin_va_start( __local_argv, __format );
  __retval = __ms_vsnprintf (__stream, __n, __format, __local_argv);
  __builtin_va_end( __local_argv );
  return __retval;
}
#endif /* !__NO_ISOCEXT */

#pragma pop_macro ("vsnprintf")
#pragma pop_macro ("snprintf")
#endif

The corresponding section in 4.6.3 as found in the Rtools for Windows
installation is:

#if !defined (__USE_MINGW_ANSI_STDIO) || __USE_MINGW_ANSI_STDIO == 0
/* this is here to deal with software defining
 * vsnprintf as _vsnprintf, eg. libxml2.  */
#pragma push_macro("snprintf")
#pragma push_macro("vsnprintf")
# undef snprintf
# undef vsnprintf
  int __cdecl vsnprintf(char * __restrict__ d,size_t n,const char *
__restrict__ format,va_list arg)
__MINGW_ATTRIB_DEPRECATED_MSVC2005 __MINGW_ATTRIB_DEPRECATED_SEC_WARN;

#ifndef __NO_ISOCEXT
  int __cdecl snprintf(char * __restrict__ s, size_t n, const char *
__restrict__  format, ...);
#ifndef __CRT__NO_INLINE
  __CRT_INLINE int __cdecl vsnprintf(char * __restrict__ d,size_t
n,const char * __restrict__ format,va_list arg)
  {
return _vsnprintf (d, n, format, arg);
  }
#endif /* !__CRT__NO_INLINE */
#endif /* !__NO_ISOCEXT */
#pragma pop_macro ("vsnprintf")
#pragma pop_macro ("snprintf")
#endif

The latter does not have a direct redefinition of the two functions. I
still don't know why the #undef calls do not work [1].

Thank you,

Avi

[1] 
https://stackoverflow.com/questions/27853225/is-there-a-way-to-include-stdio-h-but-ignore-some-of-the-functions-therein

On Thu, Jan 8, 2015 at 2:27 PM, Hin-Tak Leung
 wrote:
> Oh, I forgot to mention that besides setting AR, RANLIB and the stack probing 
> fix, you also need a very up to date binutils. 2.25 was out in december. Even 
> with that , if you linker's default is not what you are compiling for (i.e. a 
> multiarch toolchain), you need to set GNUTARGET also, i.e. -m32/-m64 is not 
> enough. Some fix to autodetect non-default targets went in after christmas 
> before the new year, but I am not brave enough to try that on a daily basis 
> yet (only tested it and reported it, then reverting the change - how gcc 
> invokes the linker is rather complicated and it is not easy to have two 
> binutils installed...)- setting GNUTARGET seems safer :-).
> Whether you need that depends on whether you are compiling for your 
> toolchain's default target architecture.
>
> AR, RANLIB, GNUTARGET are all environment variables - you set them the usual 
> way. The stack probing fix is for passing "make check", when you finish make.
>
> --
> On Thu, Jan 8, 2015 6:14 PM GMT Avraham Adler wrote:
>
>>On Thu, Jan 8, 2015 at 10:48 AM, Hin-Tak Leung
>> wrote:
>>>
>>> The r.dll crash is easy - you need to be using gcc-ar for ar, and 
>>> gcc-ranlib for ranlib. I also posted a patch to fix the check failure for 
>>> stack probing, as lto optimizes away the stack probing code, as it should.
>>>
>>> yes, lto build's speed gain is very impressive.
>>>
>>
>>
>>I apologize for my ignorance, but how would I do that? I tried by
>>changing the following in src/gnuwin32/MkRules.local:
>>
>># prefix for 64-bit: path or x86_64-w64-mingw32-
>>BINPREF64 = x86_64-w64-mingw32-gcc-
>>
>>I added the gcc- as the suffix there, but I guess that is insufficient
>>as I still get the following error using 4.9.2:
>>
>>windres -F pe-x86-64  -I../include -i dllversion.rc -o dl

Re: [Rd] New version of Rtools for Windows

[Avraham Adler]

On Fri, Jan 9, 2015 at 12:19 PM, Duncan Murdoch
 wrote:
> On 09/01/2015 10:56 AM, Henric Winell wrote:
>> On 2015-01-08 02:31, Duncan Murdoch wrote:
>>
>>> On 07/01/2015 5:20 PM, Jeroen Ooms wrote:
 On Wed, Jan 7, 2015 at 8:00 AM, Duncan Murdoch  
 wrote:
>
> This version includes only minor updates to the tools.  I indicated last 
> summer that I was hoping to update GCC from the current version 4.6.3 
> before the R 3.2.0 release, but this now looks unlikely, unless someone 
> else with experience building it can help.

 I have been looking into this a bit over the past few months, also
 with mixed success. Nevertheless, below some experiences that might be
 worth sharing.

 The guys from mingw-w64 recommended (quite strongly) to move away from
 multilib. They explained that the standard approach is to create two
 separate toolchains; one that targets win32 and the other one that
 targets win64 (both tool chains can compiled for win32). Hence the
 only difference for R would be that instead of passing "-m64" and
 "-m32", it would need to set the path to the proper compiler.

 There are several initiatives that provide very complete suites of
 precompiled mingw-w64 tools. I think the ideal scenario would be if we
 could take advantage of an existing tool chain as we do on other
 platforms, although perhaps I do not fully understand the R-specific
 requirements on the windows compiler.
>>>
>>> I feel quite strongly that we need to be able to build the toolchain,
>>> rather than relying on binaries produced by others.  We may need to
>>> customize the toolchain, or we may need to rebuild it when a bug is
>>> identified.  Lots of binary builders abandon their builds and you can't
>>> count on them to solve problems at a later date.
>>>

 One project that looks very promising is msys2 [1,2]. It has a package
 manager (port of pacman from arch linux) and comes with a pretty
 complete set of msys [3] and other [4] packages that seems quite well
 maintained.
>>>
>>> Do they post complete instructions for building?  That's what I'm
>>> looking for.  I don't want to develop a build script (I don't know how),
>>> but I would like to have one.
>>
>> Have you looked at nuwen's distro (http://nuwen.net/mingw.html)?  It's
>> up-to-date (mingw-w64 3.3.0, binutils 2.25, GCC 4.9.2, ...) and includes
>> the build scripts.
>>
>
> No, I hadn't come across that one.  It looks quite promising. Thanks!



If it helps, after installing the nuwen distro and the latest version
of Rtools (for tar), both the release R-3.1.2 and R-patched (see [1])
builds stop on Windows7 64bit with the error below, probably related
to what Hin-Tak Leung said earlier. I can't even get to the point to
see if compat.c will fail.


windres -F pe-x86-64   -i dllversion.rc -o dllversion.o
comm: file 1 is not in sorted order
Makefile.win:29: recipe for target 'Rgraphapp.def' failed
make[4]: *** [Rgraphapp.def] Error 1
makefile:120: recipe for target 'rlibs' failed
make[3]: *** [rlibs] Error 1
makefile:179: recipe for target '../../bin/x64/R.dll' failed
make[2]: *** [../../bin/x64/R.dll] Error 2
makefile:104: recipe for target 'rbuild' failed
make[1]: *** [rbuild] Error 2
makefile:14: recipe for target 'all' failed
make: *** [all] Error 2

[1] 
https://stackoverflow.com/questions/12802723/comm-file-1-is-not-in-sorted-order-when-compiling-r-source-in-windows



>
> I also have another offer of help to put this together; I'll wait to see
> how that goes before announcing it.  But having two builds is better
> than one.
>
> Duncan Murdoch
>
>
>>
>> Henric
>>
>>
>>
>>>
>>> Duncan Murdoch
>>>

 The only issue I ran into with msys2 is that it uses a different c++
 exception model (seh/dwarf) than the current Rtools (which uses sjlj).
 See also [5]. Therefore, if a library uses exceptions, we cannot use
 the current Rtools to link a static library that was created with
 msys2  [6]. I am not sure if it also be a problem the other way
 around, and if this is still the case for recent versions of
 gcc/mingw.

 Finally, Ruby has build very similar to Rtools called DevKit-mingw64
 [7] that we might be able to borrow from.


 [1] https://msys2.github.io/
 [2] 
 http://stackoverflow.com/questions/25019057/how-are-msys-msys2-and-msysgit-related-to-each-other
 [3] https://github.com/Alexpux/MSYS2-packages
 [4] https://github.com/Alexpux/MINGW-packages
 [5] 
 http://stackoverflow.com/questions/15670169/what-is-difference-between-sjlj-vs-dwarf-vs-seh
 [6] 
 http://stackoverflow.com/questions/7751640/undefined-reference-to-gxx-personality-sj0
 [7] http://rubyinstaller.org/downloads/

>>>
>>> __
>>> R-devel@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>
> 

Re: [Rd] Request for help with UBSAN and total absense of CRAN response

[Avraham Adler]

Hello, Dirk.

I have no experience with UBSAN etc., but as you mention not being
able to recreate the CRAN errors, it reminds me of the time I had
trouble getting my pacakge approved for CRAN as it passed all tets but
Win32, and I have a Win64 setup and could not test it. Have you tried
submitting the package to
 and seeing if the CRAN
errors get created there, or is the problem CRAN shows on platforms
other than Windows?

Avi

On Tue, Jan 13, 2015 at 10:30 AM, Dirk Eddelbuettel  wrote:
>
> CRAN has a package of mine in upload limbo because it failed UBSAN.
>
> I am not entirely ignorant on the topic of sanitizers and SAN / ASAN / UBSAN;
> we created not one but two Docker containers with ASAN and USBAN:
>
>https://registry.hub.docker.com/u/rocker/r-devel-san/
>https://registry.hub.docker.com/u/rocker/r-devel-ubsan-clang/
>
> as well as predecessors to them in earlier Docker repos.
>
> Yet I fail to recreate the errors reported by CRAN:
>
>
> http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN-clang-trunk/RcppAnnoy/tests/runUnitTests.Rout
>
> http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN/RcppAnnoy/tests/runUnitTests.Rout
>
> I asked politely (and twice) for help with the corresponding compiler
> configuration(s).  But CRAN is of course way above communicating with mere
> mortals such as yours truly.
>
> So I have no recourse other than to spam all of you: if anybody here has a
> working UBSAN setup which can replicate the issue seen in the (rather small)
> RcppAnnoy package?
>
> Erik (upstream for Annoy, CC'ed) and I would be most grateful.  We do not
> like being held hostage on an error report we cannot replicate and for which
> we do not receive any help (or even further communication) whatsoever.
>
> Dirk
> about to turn into yet another frustrated CRAN user
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Building rinstaller using R-devel (3.2.0-to-be) halts when trying to copy html files

[Avraham Adler]

As the build process, especially for Windows, is changing
significantly for R 3.2.0, I am trying to build R-devel in
preparation. When running `make rinstaller`, I get the following
error:

cp -p ../../../etc/x64/Makeconf R-devel/etc/x64
mkdir -p R-devel/doc
cp -p ../../../doc/CRAN_mirrors.csv R-devel/doc
mkdir -p R-devel/doc/manual/images
cp -pR ../../../doc/html R-devel/doc
cp -p ../../../doc/manual/*.html ../../../doc/manual/*.pdf \
  R-devel/doc/manual
cp: cannot stat '../../../doc/manual/*.html': No such file or directory
make[1]: *** [imagedir] Error 1
make[1]: Leaving directory `/cygdrive/f/R/R-devel/src/gnuwin32/installer'
make: *** [rinstaller] Error 2

Looking at the directories, I have "\doc\manual" and
"F:\R\R-devel\doc\html" and there are no .html files in the \manual
directory, only pdfs.

I am building on Windows 7 64 bit, so the MkRules.local is being used,
and the pertinent settings therein include:

BUILD_HTML = YES
MIKTEX = TRUE
TEXI2ANY = missing

Is it as simple as changing line 74 in the Makefile under
src/gnuwin32/installer from:

$(CP) -p $(R_HOME)/doc/manual/*.html $(R_HOME)/doc/manual/*.pdf \

to

$(CP) -p $(R_HOME)/doc/html/*.html $(R_HOME)/doc/manual/*.pdf \

Or is the problem that MIKTEX alone can no longer be used and texinfo
/must/ be installed (as is implied in
?

Thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Building rinstaller using R-devel (3.2.0-to-be) halts when trying to copy html files

[Avraham Adler]

On Mon, Jan 26, 2015 at 8:36 PM, Duncan Murdoch
 wrote:
> On 26/01/2015 5:12 PM, Avraham Adler wrote:
>> As the build process, especially for Windows, is changing
>> significantly for R 3.2.0, I am trying to build R-devel in
>> preparation. When running `make rinstaller`, I get the following
>> error:
>>
>> cp -p ../../../etc/x64/Makeconf R-devel/etc/x64
>> mkdir -p R-devel/doc
>> cp -p ../../../doc/CRAN_mirrors.csv R-devel/doc
>> mkdir -p R-devel/doc/manual/images
>> cp -pR ../../../doc/html R-devel/doc
>> cp -p ../../../doc/manual/*.html ../../../doc/manual/*.pdf \
>>   R-devel/doc/manual
>> cp: cannot stat '../../../doc/manual/*.html': No such file or directory
>> make[1]: *** [imagedir] Error 1
>> make[1]: Leaving directory `/cygdrive/f/R/R-devel/src/gnuwin32/installer'
>> make: *** [rinstaller] Error 2
>>
>> Looking at the directories, I have "\doc\manual" and
>> "F:\R\R-devel\doc\html" and there are no .html files in the \manual
>> directory, only pdfs.
>>
>> I am building on Windows 7 64 bit, so the MkRules.local is being used,
>> and the pertinent settings therein include:
>>
>> BUILD_HTML = YES
>> MIKTEX = TRUE
>> TEXI2ANY = missing
>>
>> Is it as simple as changing line 74 in the Makefile under
>> src/gnuwin32/installer from:
>>
>> $(CP) -p $(R_HOME)/doc/manual/*.html $(R_HOME)/doc/manual/*.pdf \
>>
>> to
>>
>> $(CP) -p $(R_HOME)/doc/html/*.html $(R_HOME)/doc/manual/*.pdf \
>>
>> Or is the problem that MIKTEX alone can no longer be used and texinfo
>> /must/ be installed (as is implied in
>> <http://cran.r-project.org/doc/manuals/r-devel/R-admin.html#Building-the-manuals>?
>
> Yes, for a complete build you need the texi2any program (and Perl).  R
> will still build well enough to run, but the help system is incomplete,
> and will require an Internet connection to access the manuals.
>
> Were you interested in distributing the version without the manuals, or
> is it just that you haven't updated your Rtools to include texi2any?
>
> Duncan Murdoch
>

I was afraid of that. I had updated Rtools with everything (Perl, ICU,
etc.) but was afraid texinfo would interfere with MikTex; my mistake.
Fixing that worked, thank you very much. In general, even though I
don't distribute binaries to others, I prefer to run through the build
all the way to risntaller and then run check-all to make sure
everything is working properly. Also, it allows me to use the
installer to install a production version of R to one subdirectory and
keep mu

Two points if I may:

1) The MkRules has the flag TEX2ANY and the texinfo zip has a flag
MAKEINFO which seems to require the same statement. I put both in the
MkRules.local, but which one should be used?
2) For windows users "/path/to/perl" may be a bit misleading. Most of
the other entries require the subdirectory name. This one seems to
require the full name of the executable (e.g.
"F:/Strawberry/perl/bin/perl.exe").

Once again, thank you,

Avi

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel