[Rd] LCONS undefined for R_NO_REMAP

[Jeroen Ooms]

Rinternals.h has:

#define CONS(a, b) cons((a), (b))
#define LCONS(a, b) lcons((a), (b))

However these are undefined when we compile with -DR_NO_REMAP. Maybe
it's safer to define these using Rf_cons() and Rf_lcons() instead?

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Setting the path to Rtools for package compilation on Windows

[Jeroen Ooms]

On Tue, Feb 13, 2018 at 7:45 AM, Peter Langfelder
 wrote:
> Hi all,
>
> I'm trying to set up the Windows Rtools toolset for building packages
> with compiled code. I installed for Windows R-3.4.3 from CRAN and
> installed Rtools-3.4 in a custom location M:\R\R-3.4.3 and
> M:\R\Rtools-3.4
>
> Following the instructions, in shell, I set
> Path=M:\R\Rtools-3.4\bin;M:\R\Rtools-3.4\gcc-4.6.3\bin;M:\R\R-3.4.3\bin;...
> (the ... are other paths irrelevant for R/Rtools).

Thanks for your question. Your logs show that 'gcc' is not found. As
of R 3.3 you need to set the path to the compiler using the BINPREF
variable. This is because we ship two separate versions of gcc, one
targeting win32 and one targeting win64. I am not sure what your
rtools installation looks like, but could you try setting this
environment variable:

  BINPREF="M:/R/Rtools-3.4/mingw_$(WIN)/bin/"

I think this will do the job.

On Tue, Feb 13, 2018 at 10:21 AM, Tomas Kalibera
 wrote:
> Thanks for the report - this has been already reported as bug 17376, it is
> caused by scripts that build the Windows binaries and by now has been fixed
> in R-patched and R-devel snapshot builds. So as a solution that works now I
> would recommend using R-patched.

This issue is unrelated, I doubt your advice will solve anything. The
only thing that 17376 does is add c:/rtools/bin to the default path.
But this dir does not exist for this user, so it is ignored by windows
when searching the PATH.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Compilation gets stuck on Windows 64

[Jeroen Ooms]

On Tue, Feb 13, 2018 at 12:22 PM, Indrajit Sen Gupta
 wrote:
> Hi Avraham,
>
> I tried with the patched version. The same error message.
>
> gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o dynload.o
> editor.o embeddedR.o extra.o malloc.o opt.o pager.o preferences.o psignal.o
> rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o system.o dos_wglob.o
> dllversion.o ../main/libmain.a ../appl/libappl.a ../nmath/libnmath.a
> getline/gl.a ../extra/xdr/libxdr.a ../extra/intl/libintl.a
> ../extra/trio/libtrio.a ../extra/tzone/libtz.a ../extra/tre/libtre.a
> -fopenmp -L. -lgfortran -lquadmath -lRblas -L../../lib -lRgraphapp -lRiconv
> -lcomctl32 -lversion -L"D:/R64/extsoft"/lib/x64 -lpcre -lz -lbz2 -llzma
> -L"D:/home/ICU"/lib/x64 -lsicuin -lsicuuc -lsicudt -lstdc++
> D:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe:
> cannot find -lRgraphapp
> collect2.exe: error: ld returned 1 exit status
> make[4]: *** [R.dll] Error 1
> make[3]: *** [../../bin/x64/R.dll] Error 2
> make[2]: *** [rbuild] Error 2
> make[1]: *** [all] Error 2
> make: *** [distribution] Error 2
>
>
> Would it be possible for you to share your MkRules.local and Makefile.win
> files?


Hi Indrajit

As somebody above already mentioned, the full build script as well as
MkRules.local that we use for the CRAN releases of R for windows are
available from https://github.com/rwinlib/base

As is explained in the repository readme, if you have the required
dependencies (rtools, miktex innosetup, strawberry perl) all you need
to do is run the build-r-devel.bat script from the root of the
repository.

Once you got this to work, you can adapt it to your needs.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Compilation gets stuck on Windows 64

[Jeroen Ooms]

On Tue, Feb 13, 2018 at 2:18 PM, Indrajit Sen Gupta
 wrote:
> In the file MkRules.local.in, I see the line: USE_ATLAS = NO which I believe
> needs to be changed to YES. But how do I specify the BLAS file
> libopenblas_haswell-r0.2.20.a and its location?

I have never done this, but a good starting point is searching the R
source code for how this works:
https://github.com/wch/r-source/search?utf8=✓&q=USE_ATLAS

>From the extra/blas/Makefile.win file it looks like you may need to
rename libopenblas_haswell-r0.2.20.a to libatlas.a and point to this
directory via the variable ATLAS_PATH in your MkRules.local.in?

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Setting the path to Rtools for package compilation on Windows

[Jeroen Ooms]

On Tue, Feb 13, 2018 at 7:58 PM, Peter Langfelder
 wrote:
> On Tue, Feb 13, 2018 at 2:20 AM, Jeroen Ooms  wrote:
>
>> Thanks for your question. Your logs show that 'gcc' is not found. As
>> of R 3.3 you need to set the path to the compiler using the BINPREF
>> variable. This is because we ship two separate versions of gcc, one
>> targeting win32 and one targeting win64. I am not sure what your
>> rtools installation looks like, but could you try setting this
>> environment variable:
>>
>>   BINPREF="M:/R/Rtools-3.4/mingw_$(WIN)/bin/"
>>
>> I think this will do the job.
>
> Thanks, that indeed did the trick. May I suggest that this hint be
> also included in "REMAINING TASKS" section of the file Rtools.txt that
> is part of the Rtools distribution?  The R installation manual does
> mention BINPREF and BINPREF64 but I missed that part...

Glad it worked. Actually BINPREF64 only exists when building R itself.
For the R user there is no BINPREF64. There is only BINPREF which has
to point to a 32bit gcc when R runs in 32bit, and a 64bit compiler
when R runs in 64bit. But if you compile R packages you need both at
the same time. Hence the "$(WIN)" variable in the BINPREF above.

Yes the rtools setup is a bit convoluted, and we're currently
discussing ways to improve this. Thanks for your suggestion.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Problem with R_registerRoutines

[Jeroen Ooms]

On Windows this warning may be a false positive if R cannot find
"objdump.exe" which is required for this check. I think this is
actually a bug in R because it should be looking for "objdump.exe"
inside BINPREF (where gcc is) rather than on the PATH.

Can you check if you get the same warning if you upload the package to
https://win-builder.r-project.org ?






On Fri, Feb 23, 2018 at 10:28 AM,   wrote:
> Dear list,
>
> I am trying to update a package to pass the CRAN-checks.
> But I am struggling with the following note:
>
> File 'psgp/libs/i386/psgp.dll':
>   Found no calls to: 'R_registerRoutines', 'R_useDynamicSymbols'
> File 'psgp/libs/x64/psgp.dll':
>   Found no calls to: 'R_registerRoutines', 'R_useDynamicSymbols'
>
> It is good practice to register native routines and to disable symbol
> search.
>
>
> I did already run:
> tools::package_native_routine_registration_skeleton(".")
> This gave me some code, including a function R_init_psgp, which includes 
> calls to the functions above, and also the names of the C++ functions to be 
> called from R.
> I first saved this code in registerDynamicSymbol.c and added .registration = 
> TRUE to useDynLib in the NAMESPACE file.
> I still get the error above. As I saw that the file has different names in 
> other packages, I have also tried to save it psgp_init.c, and in init.cpp, 
> still with the same error message.
>
> I have read the relevant part of the R extensions manual, but could not find 
> anything that could help me with this problem.
> I have had a look at the similar files in other packages (including one of my 
> own, which works), and the initialization seems fine to me.
> There is surely something I have overlooked, is anyone able to give me a hint 
> to where I might look? The code is in C++, not sure if that could have 
> anything to do with the problem?
>
> Thanks,
> Jon
>
>
>
>
>
>
>
>
>
>
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] data.table not available as win binary for R 3.5

[Jeroen Ooms]

On Tue, Apr 24, 2018 at 7:26 AM, Joris Meys  wrote:
>
> Dear all,
>
> to my astonishment data.table cannot be installed on R 3.5 Windows. When
> checking the package page, the Windows binary is available for download.


The package check page for data.table shows that is currently failing
CMD check. As a precaution, CRAN does not publish binaries for
packages that do not pass check, so I think this is why it seems
unavailable.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] download.file does not process gz files correctly (truncates them?)

[Jeroen Ooms]

On Thu, May 3, 2018 at 2:42 PM, Henrik Bengtsson
 wrote:
> Use mode="wb" when you download the file. See
> https://github.com/HenrikBengtsson/Wishlist-for-R/issues/30.
>
> R core, and others, is there a good argument for why we are not making this
> the default download mode? It seems like a such a simple fix to such a
> common "mistake".

I'd like to second this feature request. This default behaviour is
unexpected and often leads to r scripts that were written on
mac/linux, to produce corrupted files on windows, checksum mismatches,
etc.

Even for text files, the default should be to download the file as-is.
Trying to "fix" line-endings should be opt-in, never the default.
Downloading a file via a browser or ftp client on windows also doesn't
change the file, why should R?


On Thu, May 3, 2018 at 3:02 PM, Duncan Murdoch  wrote:
> Many downloads are text files (HTML, CSV, etc.), and if those are downloaded
> in binary, a Windows user might end up with a file that Notepad can't
> handle, because it would have Unix-style line endings.

True but I don't think this is relevant. The same holds e.g. for the R
files in source packages, which also have unix line endings. Most
Windows users will use an actual editor that understands both types of
line endings, or can convert between the two.

Downloading-file should do just that.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] potential Bug in R 3.5.0

[Jeroen Ooms]

On Sun, Jun 24, 2018 at 8:41 PM, Jochen Wirsing  wrote:
> Hello,
> I am using R under Win7 and Antergos. After installing R 3.5.0 I ran into
> several issues regarding gfortran not working properly and packages not
> installing properly when requiring dependencies (which is quite often the case
> when you have to reinstall all packages for the new R version).
> Unfortunately, I cannot reproduce the gfortran issue at the moment (occuring
> under Antergos), but regarding the depency-issue, https://pastebin.com/
> 0nU5n3pH shows one instance of the problem, when using "tidyverse". Adding
> "dependencies = TRUE" does not help.

It looks like R thinks those packages (‘dbplyr’, ‘rvest’, ‘xml2’) are
already installed, but is unable to load them. I have never used
'Antergos' but could it be that those packages were installed with R
3.4 and therefore stopped working after you upgraded to R 3.5? You
need to rebuild all R packages after you update to R. The easiest way
is to run:

  update.packages(ask = F, checkBuilt = T)

Also check your .libPaths() to make sure that there are no old
packages left behind in older directories.

If this is not the issue, try finding out where those packages live
using e.g. find.packages("xml2") and see what error you get when
trying to load them with library("xml2").

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] undefined symbol: cholmod_factorize_p

[Jeroen Ooms]

On Fri, Jul 6, 2018 at 11:57 PM, Göran Broström  wrote:
> I am installing R_3.5.1 from source on ubuntu 18.04, and 'config' + 'make'
> gives me (at the end)

Probably something went wrong earlier in the build when compiling the
Matrix package. However it's much easier to install R form the
binaries:

  sudo add-apt-repository -y ppa:marutter/rrutter3.5
  sudo apt-get update
  sudo apt-get install r-base

If you really want to build from source, have a look at which flags
and dependencies are used in the r-base source package on
Debian/Ubuntu.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] substitute() on arguments in ellipsis ("dot dot dot")?

[Jeroen Ooms]

On Sun, Aug 12, 2018 at 10:00 PM, Henrik Bengtsson
 wrote:
> Hi. For any number of *known* arguments, we can do:
>
> one <- function(a) list(a = substitute(a))
> two <- function(a, b) list(a = substitute(a), b = substitute(b))
>
> and so on. But how do I achieve the same when I have:
>
> dots <- function(...) list(???)
>
> I want to implement this such that I can do:
>
>> exprs <- dots(1+2)
>> str(exprs)
> List of 1
>  $ : language 1 + 2
>
> as well as:
>
>> exprs <- dots(1+2, "a", rnorm(3))
>> str(exprs)
> List of 3
>  $ : language 1 + 2
>  $ : chr "a"
>  $ : language rnorm(3)
>
> Is this possible to achieve using plain R code?

You could use match.call, for example:

  dots <- function(...) match.call(expand.dots = FALSE)[['...']]

Note that this returns a pairlist, so if you want an ordinary list you
should wrap it in as.list()

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] segfault issue with parallel::mclapply and download.file() on Mac OS X

[Jeroen Ooms]

On Thu, Oct 4, 2018 at 6:12 PM Tomas Kalibera  wrote:
>
>
> Thanks for the report, but unfortunately I cannot reproduce on my system
> (either macOS nor Linux, from the command line) to debug. Did you run
> this in the command line version of R?

It depends on which version of MacOS that you are using and
specifically which TLS back-end curl has been configured with. When
libcurl uses DarwinSSL, it may crash when opening HTTPS connections in
a fork because CoreFoundation is not fork-safe. OTOH when using
OpenSSL or LibreSSL for TLS, you usually get away with forking (though
it's still bad practice).

The standard version of libcurl that ships with MacOS was using
CoreFoundation until 10.12 but starting 10.13 they switched to
LibreSSL in order to support HTTP/2. See curl --version or
curl::curl_version() for your local config. Don't count in this
though, Apple might switch back to the fork-unsafe DarwinSSL once they
support ALPN, which is needed for HTTP/2.

As Gabor already suggested, libcurl has built-in systems for
concurrent connections. The curl package exposes this via multi_add
function. Not only is this safer than forking, it will be much faster
because it takes advantage of HTTP keep-alive and when supported it
uses HTTP2 multiplexing which allows efficiently performing thousands
of concurrent HTTPS requests over a single TCP connection.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Change windows installer default to only install 64 bit R

[Jeroen Ooms]

On Fri, Nov 9, 2018 at 5:47 PM Jim Hester  wrote:
> We can remove much of this ambiguity by changing the default choice in
> the installer to only install the 64 bit version in the installer. If
> users do need the 32 bit version it is still simple for them to
> install it by checking the appropriate box during installation.

I tested the patch and confirm that this works as intended. By
default, the R installer will leave the "32-bit Files" unchecked on
Windows 64 machines.

I also think this proposal makes sense. Most users do not need 2
versions of R on their machine, and it is a common source of problems.
I agree with Jim that installing the 32bit version of R on 64bit
Windows would better be opt-in rather than default.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OfficeScan deletes Rterm as malware

[Jeroen Ooms]

On Thu, Dec 20, 2018 at 2:52 PM Roland Fuß  wrote:
>
> I'm not sure if the problem is actually with R but thought I should
> report this anyway.
>
> After Peter's email regarding the R 3.5.2 release today, I installed the
> Windows version right away (directly from CRAN and not from a mirror).
>
> Unfortunately, my institute's AV sofware TrendMicro OfficeScan 12.0.5147
> Service Pack 1 stops and deletes Rterm.exe when running Rcmd.exe INSTALL
> with a local source package. It reports "OfficeScan detected a Behavior
> Monitoring policy violation and blocked the offending process(es)." and
> "Unauthorized File Encryption" by Rterm.exe.

I can't think of anything that has changed between R 3.5.1 and 3.5.2.
What does "Behavior Monitoring policy violation" mean? Does it say
what sort of policy?

A quick search reveals that the TrendMicro "Unauthorized File
Encryption" message is a frequent false positive for many different
software programs, including this thread at the rstudio forum:
https://support.rstudio.com/hc/en-us/community/posts/208171047-rsession-exe-flagged-as-virus

Perhaps it's just because you're one of the first people to install
this version, and your AV might use some "smart" learning system such
that the false positive will automatically disappear after a few more
people have installed and whitelisted the new R binaries.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Issue when building R-parched_2018-12-26

[Jeroen Ooms]

On Wed, Dec 26, 2018 at 8:24 PM Avraham Adler 
wrote:

> Hello.
>
> Building R-patched from source on Win64 using current Rtools 3.5.0.4, I’m
> getting the following notes indicating an extra left brace somewhere; or is
> the problem on my end?
>

This is a warning (not an error) in the texinfo script that has started to
appear with the latest version of perl. It is safe to ignore the warning,
but it has been fixed in rtools40:
https://github.com/r-windows/rtools-base/commit/d1631cc9646121f580742a29b2ee3c3b8a569685

I haven't back-ported this to rtools35 because I'm still hopeful that we
will upgrade to rtools40, soonish...

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Trying to compile R 3.5.2 - 32 bit R - on Windows 10 64 bit - with ICU support

[Jeroen Ooms]

On Sat, Feb 16, 2019 at 11:13 AM Andre Mikulec
 wrote:
>
> Hi,
>
> I am trying to compile R with ICU support.

Are you aware that the default R for Windows installation from CRAN
has ICU support?

> I am following
> https://cran.r-project.org/doc/manuals/R-admin.html#Building-from-source

These instructions are unfortunately outdated. The scripts and
instructions used to build the current version of R for Windows can be
found here: https://github.com/rwinlib/base#readme

> I have downloaded and extracted
> https://www.stats.ox.ac.uk/pub/Rtools/goodies/ICU_531.zip

That build of ICU was for the previous toolchain (up to R 3.2.5). You
need to use the ICU build which is included with Rtools (if you
checked the icu checkbox during the Rtools installation). The scripts
mentioned above will do this automatically.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Is libtiff >= 4.0.0 now required by R for TIFF support?

[Jeroen Ooms]

On Mon, Feb 25, 2019 at 11:11 AM Bjørn-Helge Mevik
 wrote:
>
> We recently discovered that since R 3.3.0, on our CentOS 6 based
> cluster, R gets built without TIFF support.  The last version where TIFF
> support was built, was 3.2.5.

It looks like the official R from EPEL6 also lacks tiff support:

  yum install epel-release
  yum install R-devel
  R -e 'capabilities()'

> capabilities()
   jpeg pngtiff   tcltk X11aqua
   TRUETRUE   FALSETRUE   FALSE   FALSE
   http/ftp sockets  libxmlfifo  cledit   iconv
   TRUETRUETRUETRUE   FALSETRUE
NLS profmem   cairo ICU long.double libcurl
   TRUE   FALSETRUETRUETRUETRUE

So your hypothesis that autoconf doesn't find libtiff < 4 is probably correct.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R 3.5.3 and 3.6.0 alpha Windows bug: UTF-8 characters in code are simplified to wrong ones

[Jeroen Ooms]

On Wed, Apr 10, 2019 at 12:19 PM Tomáš Bořil  wrote:
>
> Minimalistic example:
> Let's type "ř" (LATIN SMALL LETTER R WITH CARON) in RGui console:
> > "ř"
> [1] "r"
>
> Although the script is in UTF-8, the characters are replaced by
> "simplified" substitutes uncontrollably (depending on OS locale). The
> same goes with simply entering the code statements in R Console.
>
> The problem does not occur on OS with UTF-8 locale (Mac OS, Linux...)

I think this is a "feature" of win_iconv that is bundled with base R
on Windows (./src/extra/win_iconv). The character from your example is
not part of the latin1 (iso-8859-1) set, however, win-iconv seems to
do so anyway:

> x <- "\U00159"
> print(x)
[1] "ř"
> iconv(x, 'UTF-8', 'iso-8859-1')
[1] "r"

On MacOS, iconv tells us this character cannot be represented as latin1:

> x <- "\U00159"
> print(x)
[1] "ř"
> iconv(x, 'UTF-8', 'iso-8859-1')
[1] NA

I'm actually not sure why base-R needs win_iconv (but I'm not an
encoding expert at all). Perhaps we could try to unbundle it and use
the standard libiconv provided by the Rtools toolchain bundle to get
more consistent results.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R 3.5.3 and 3.6.0 alpha Windows bug: UTF-8 characters in code are simplified to wrong ones

[Jeroen Ooms]

On Wed, Apr 10, 2019 at 5:45 PM Duncan Murdoch  wrote:
>
> On 10/04/2019 10:29 a.m., Yihui Xie wrote:
> > Since it is "technically easy" to disable the best fit conversion and
> > the best fit is rarely good, how about providing an option for
> > code/package authors to disable it? I'm asking because this is one of
> > the most painful issues in packages that may need to source() code
> > containing UTF-8 characters that are not representable in the Windows
> > native encoding. Examples include knitr/rmarkdown and shiny. Basically
> > users won't be able to knit documents or run Shiny apps correctly when
> > the code contains characters that cannot be represented in the native
> > encoding.
>
> Wouldn't things be worse with it disabled than currently?  I'd expect
> the line containing the "ř" to end up as NA instead of converting to "r".

I don't think it would be worse, because in this case R would not
implicitly convert strings to (best fit) latin1 on Windows, but
instead keep the (correct) string in its UTF-8 encoding. The NA only
appears if the user explicitly forces a conversion to latin1, which is
not the problem here I think.

The original problem that I can reproduce in RGui is that if you enter
 "ř" in RGui, R opportunistically converts this to latin1, because it
can. However if you enter text which can definitely not be represented
in latin1, R encodes the string correctly in UTF-8 form.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Inno Setup 6.0.2 fails before creating exe file on Windows (R-3.6.0)

[Jeroen Ooms]

Multiple people have now reported that the R installer does not build
with InnoSetup 6 (released last week). The inno log shows this error:

  Error on line 12 in src\gnuwin32\installer\R.iss: Minimum NT version
specified by MinVersion must be at least 6.0. (Windows 2000/XP/Server
2003 are no longer supported.)
  Compile aborted.

Hence to fix this, the "MinVersion" parameter in
src/gnuwin32/installer/header1.iss should be increased from 0,5.0 to
0,6.0. I confirmed this resolves the problem. Could somebody commit
this please?

See also: http://www.jrsoftware.org/is6help/index.php?topic=setup_minversion



On Sun, Apr 28, 2019 at 11:07 AM Avraham Adler  wrote:
>
> I am working on compiling R-3.6.0 for Windows 10 64bit using rtools40
> (beta 11). I had also installed the most recent update of Inno setup,
> which is now 6.0.2.With that version, `make risntaller` fails at the
> call to ""C:/R/Inno/iscc" R.iss > R-3.6.0.log 2>&1" and just exits,
> pointing to line 175 of the makefile which is:
>
> $(RPREFIX)-win.exe: R.iss
> "$(ISDIR)/iscc" R.iss > $(RPREFIX).log 2>&1
>
> Reinstalling Inno Setup 5.6.1 does allow the exe file to be created.
>
> Thank you,
>
> Avi
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] writing Unicode text to the Windows clipboard

[Jeroen Ooms]

On Fri, May 24, 2019 at 12:06 AM Jennifer Bryan  wrote:
>
> Hello,
>
> I'm interested in moving text from and to the clipboard that cannot
> necessarily be represented in the native encoding. So, really, this is
> about Windows.
>
> I can successfully read from the clipboard by specifying the format that
> corresponds to unicode text.
>
> From R >=2.7.0, it seems you should also be able to write unicode text
> to the Windows clipboard.
>
> https://github.com/wch/r-source/blob/5a156a0865362bb8381dcd69ac335f5174a4f60c/src/gnuwin32/CHANGES0#L535-L536

Thanks!

I tested this and can confirm that this patch works. With this change,
any (non ascii) unicode text now properly copies and pastes from/to
the Windows clipboard.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Offer zip builds

[Jeroen Ooms]

On Tue, Jun 4, 2019 at 5:40 PM Steven Penny  wrote:
>
> Theres nothing nefarious here. It would allow people to use the R environment
> without running an installer. If someone is a new user they may want to try
> R out, and installers can be invasive as they commonly:
>
> - copy files to install dir
> - copy files to profile dir
> - set registry entries
> - set environment variables
> - set start menu entries
>
> and historically uninstallers have a bad record of reverting these changes.
> should not put this burden upon new users or even having them resort to 
> virtual
> machine to avoid items above. having a ZIP file allows new users to run the
> R environment, then if they like it perhaps they can run the installer going
> forward.

This is a valid suggestion, but probably impossible to do reliably.
Most installers (the R one is completely open source btw) perform
those steps for a reason. It is great if software can be installed
simply by extracting a zip file somewhere, but if this is what you
desire, you're using the wrong operating system.

We only offer official installation options that work 100% reliably
and I don't think this can be accomplished with a zip file. For
example a zip file won't be able to set the installation location in
the registry, and hence other software such as RStudio won't be able
to find the R installation. Also a zip installation might mix up
package libraries from different R versions (which is bad), or users
might expect they can upgrade R by overwriting their installation with
a new zip (also bad). Hence I'm afraid offering such alternative
installation options would open a new can of worms with bug reports
from Windows users with broken installations, or packages that don't
work as expected.

As for alternatives, 'rportable' and 'innoextract' have already been
mentioned if you really just want to dump the files from the
installer, if that works for you. Another popular option to install
(any) Windows software without manually running installers is using
chocolatey, for example:

  choco install miktex
  choco install r.project

This will still indirectly use official installers, but the installers
have been verified as "safe" by external folks and the installation is
completely automated. Perhaps that's another compromise you could live
with.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Rtools contains Python interpreter(s), and six copies?

[Jeroen Ooms]

On Fri, Aug 2, 2019 at 12:12 AM Abby Spurdle  wrote:
>
> I've just discovered that Rtools (on Windows) contains Python
> interpreter(s).

A minimal build of python was included as a dependency of gdb in
Rtools 3.3 and up.

> I'm assuming that Python is required to build R packages, on all operating
> systems.

Please don't assume but read the documentation (preferably before posting).

> Also, by my count, Rtools contains six Python interpreters.
> I've miscounted, I hope...

We have a dual toolchain so there is a 32 and a 64 bit build of
python2.7.exe. The other names (python.exe and python2.exe) are
symlinks that python automatically creates. These are not in any
location that is ever on the path so nothing to worry about.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Can't build R 3.6.1 or 3.5.3 on Ubuntu 18.04 LTS

[Jeroen Ooms]

On Fri, Nov 22, 2019 at 2:58 PM Saren Tasciyan  wrote:
>
> Hi,
>
> I am trying to build a more recent version (3.6.1 or 3.5.3) of R on
> Ubuntu 18.04 LTS.

Are you aware you can get up-to-date binaries either from PPA or CRAN:

  sudo add-apt-repository ppa:marutter/rrutter
  sudo apt-get install r-base-dev

> and I ran ./configure
> During "make", I get the following error:
> platform.o: In function `do_eSoftVersion':
> /home/user/Desktop/R-3.6.1/src/main/platform.c:3129: undefined reference
> to `u_getVersion_58'

Such linking errors indicate you are not linking against the correct
library that you compiled against. In this case, there seems to be a
version conflict because you compiled against version 58 of ICU, but
Ubuntu Bionic actually has ICU60:
https://packages.ubuntu.com/bionic/libicu-dev

So the question is how do you end up with ICU58 headers? I am guessing
your either installed a custom libicu in /usr/local/, or you upgraded
from an older version of ubuntu but haven't upgraded libicu-dev.
Either way the solution is to clean up your system and make sure there
is only one version of libicu.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Development version of R fails tests and is not installed

[Jeroen Ooms]

On Sat, Feb 8, 2020 at 9:27 AM Berwin A Turlach
 wrote:
>
> G'day all,
>
> I have daily scripts running to install the patched version of the
> current R version and the development version of R on my linux box
> (Ubuntu 18.04.4 LTS).
>
> The last development version that was successfully compiled and
> installed was "R Under development (unstable) (2020-01-25 r77715)" on
> 27 January.  Since then the script always fails as a regression test
> seems to fail.  Specifically, in the tests/ subdirectory of my build
> directory I have a file reg-tests-1d.Rout.fail which ends with:
>
> > ## more than half of the above were rounded *down* in R <= 3.6.x
> > ## Some "wrong" test cases from CRAN packages (partly relying on wrong R <= 
> > 3.6.x behavior)
> > stopifnot(exprs = {
> + all.equal(round(10.7775, digits=3), 10.778, tolerance = 1e-12) # even 
> tol=0, was 10.777
> + all.equal(round(12345 / 1000,   2), 12.35 , tolerance = 1e-12) # even 
> tol=0, was 12.34 in Rd
> + all.equal(round(9.18665, 4),9.1866, tolerance = 1e-12) # even 
> tol=0, was  9.1867
> + })
> Error: round(10.7775, digits = 3) and 10.778 are not equal:
>   Mean relative difference: 9.27902e-05
> Execution halted
>
> This happens while the 32bit architecture is installed,  which is a bit
> surprising as I get the following results for the last installed
> version of R's development version

There are two independent, but slightly related issues here:

First, as Martin already explained, the round() function was recently
improved, and some very strict tests were added to confirm the new
behavior. That explains why you see different round() results in R 4.0
from R 3.6.2. The bugzilla thread explains why:
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17668

The second issue has to do with numeric precision on 32-bit systems,
which is why I think you are getting this error. We ran into the same
problem on Windows where results on 32-bit are slightly off, including
(but not limited to) edge-cases in rounding. This has always been the
case, but the 32-bit inaccuracies have increased for recent versions
of GCC.

In general, the main difference in float precision between i686 and
x86_64 could come from whether it uses x87 (with 80 bit floats as
intermediates, as long as all intermediates are stored in registers)
or sse2 for general math. Depending on what the tests do, you can get
test failures (i.e. different results) if intermediates use different
precision, if the test reference is calculated assuming rounding all
intermediates to a certain length between each step.

The solution: to get the same results on 32-bit as on 64-bit, you need
to build R with these extra gcc flags: -mfpmath=sse -msse2. As
explained in 
https://gcc.gnu.org/onlinedocs/gcc-8.3.0/gcc/x86-Options.html#x86-Options
the -mfpmath=sse is the default for x86-64 but not for i686. As of
r77719 we have made sse the default on Windows and now we get
consistent results on 32-bit and 64-bit, including the round() edge
cases.

I think the intention was to add something similar in R's autoconf
script to enable sse on 32-bit unix systems, but seemingly this hasn't
happened. For now I think you should be able to make your 32-bit
checks succeed if you build R with CFLAGS=-mfpmath=sse -msse2.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Rtools and R 4.0.0?

[Jeroen Ooms]

On Mon, Apr 6, 2020 at 9:15 AM Prof Brian Ripley  wrote:
>
> On 02/04/2020 05:35, Kevin Ushey wrote:
> > Hello,
> >
> > Has a decision been made yet as to whether R 4.0.0 on Windows is going
> > to be built using the new gcc8 toolchain (described at
> > https://cran.r-project.org/bin/windows/testing/rtools40.html)?
>
> Short answer: 'no'.
>
> >>From the sidelines, I can see that the toolchain is being used to
> > build and test packages on CRAN; if there are any remaining issues
> > that I can help to try and run down (either in R or any CRAN packages)
> > I'd be happy to try and help.
>
> It is still under consideration, but resource constraints are biting
> hard as people's lives get more complicated.

Given the amount of work that has gone into this and the importance of
an up-to-date toolchain for maintaining well-functioning and current
system libraries and R packages, I think it should be considered to
delay the R 4.0 release over this until everyone is ready.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Rtools and R 4.0.0?

[Jeroen Ooms]

On Mon, Apr 6, 2020 at 9:39 AM Gabriel Becker  wrote:
>
> Hi Kevin,
>
> On Wed, Apr 1, 2020 at 9:36 PM Kevin Ushey  wrote:
>
> > Hello,
> >
> > FWIW, I'm excited at the prospect at seeing a new toolchain for
> > Windows, since it would imply support for C++17 and so it would become
> > easier for CRAN packages to depend on the newer C++ standard.
> >
>
> One thing to keep in mind (having been the R installation owner in such a
> place for multiple years) is that many coproprate or otherwise controlled
> compute environments may not have access to a c++17 compiler on their
> servers so making it easy for packages to rely on that is not purely
> beneficial to all parts of the R community.

No, you're missing an important point here. On Windows, the toolchain
version is tied to the version of R and we try to keep supporting at
least one or two previous versions of R. So this means we always need
to support the legacy toolchain for a while as well.

Hence if we switch Windows to gcc-8 for R 4.0, we still rely on
gcc-4.9 for continued support of R 3.3-3.6. This lag is what is making
the maintenance of windows system libraries painful, and why we need
to plan ahead. This is different from Linux where version of the
compiler is given by the OS and not tied to the version of R.

If we would miss the boat again, and R 4.0 on Windows would stick with
gcc-49, this means we need to keep supporting gcc-49 as long as we
want to support R-4.0, which is at least 2022 or 2023. This would be
pretty bad. Even currently the latest versions of important system
libraries used by R packages (e.g. the gdal stack) require recent
compilers and cannot be built anymore with gcc-49. As more C++
projects are adopting C++14/17, we can no longer update these system
libraries, missing out on all upstream fixes and advances. This would
seriously decrement the quality of the R ecosystem.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] missing binaries in R-devel windows snapshot 78175

[Jeroen Ooms]

On Thu, Apr 9, 2020 at 12:44 PM Bravington, Mark (Data61, Hobart)
 wrote:
>
> The "r-devel snapshot build" 78175 on Windows--- a dot-exe installer--- seems 
> to be missing a couple of files in its bin/i386 folder: Rterm.exe and 
> Rgui.exe. Both are present in its bin/x64 folder (and in the i386 folder for 
> current R).

I just tested this and as far as I can tell there are no missing
files. Are you sure there isn't a local problem with your system
permissions or antivirus that is removing the files?

Try to verify the md5 of the installer; some enterprise firewalls are
tampering with downloads:

  
openssl::md5(url('https://cran.r-project.org/bin/windows/base/R-devel-win.exe'))
  readLines('https://cran.r-project.org/bin/windows/base/md5sum.txt.R-devel')

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] missing binaries in R-devel windows snapshot 78175

[Jeroen Ooms]

On Fri, Apr 10, 2020 at 2:42 AM Bravington, Mark (Data61, Hobart)
 wrote:
>
> > On Thu, Apr 9, 2020 at 12:44 PM Bravington, Mark (Data61, Hobart)
> >  wrote:
> > >
> > > The "r-devel snapshot build" 78175 on Windows--- a dot-exe installer--- 
> > > seems to be missing a couple of files in its bin/i386 folder: Rterm.exe 
> > > and Rgui.exe. Both are present in its bin/x64 folder (and in the i386 
> > > folder for current R).
>
> From: Jeroen Ooms 
> Sent: Thursday, 9 April 2020 21:32
> To: Bravington, Mark (Data61, Hobart)
> Cc: R-Devel-2
> Subject: Re: [Rd] missing binaries in R-devel windows snapshot 78175
>
>
> > I just tested this and as far as I can tell there are no missing
> > files. Are you sure there isn't a local problem with your system
> > permissions or antivirus that is removing the files?
>
> You're quite right--- my apologies. I've been Cylanced (without it having the 
> politeness to actually tell me, g). I tried again with the 78172 build 
> that you used, and the two dot-exe files are there immediately after 
> installation--- but 10 minutes later they're gone. One question below about 
> the md5 stuff, though:
>
> > Try to verify the md5 of the installer; some enterprise firewalls are
> > tampering with downloads:
>
> >   
> > openssl::md5(url('https://cran.r-project.org/bin/windows/base/R-devel-win.exe'))
> >   
> > readLines('https://cran.r-project.org/bin/windows/base/md5sum.txt.R-devel')
>
> Thanks for the tip. I did that, and both give the same output. But:
>
>  - If I run 'md5sum' on my local copy of the 78182 installer (ie the file 
> I've just downloaded) I again get the same signature as per your two lines. 
> (It's not the firewall, it's Cylance.)
>
>  - Yet if I run 'openssl::md5( )' I get a different 
> signature!

I think you're calculating the md5 of the filename, not the file. You
need to use:

  openssl::md5(file("R-devel-win.exe"))

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] missing binaries in R-devel windows snapshot 78175

[Jeroen Ooms]

On Fri, Apr 10, 2020 at 9:49 AM Martin Maechler
 wrote:
>
> >>>>> Jeroen Ooms
> >>>>> on Fri, 10 Apr 2020 08:54:39 +0200 writes:
>
> > On Fri, Apr 10, 2020 at 2:42 AM Bravington, Mark (Data61,
> > Hobart)  wrote:
> >>
> >> > On Thu, Apr 9, 2020 at 12:44 PM Bravington, Mark
> >> (Data61, Hobart) > 
> >> wrote:
> >> > >
> >> > > The "r-devel snapshot build" 78175 on Windows--- a
> >> dot-exe installer--- seems to be missing a couple of
> >> files in its bin/i386 folder: Rterm.exe and
> >> Rgui.exe. Both are present in its bin/x64 folder (and in
> >> the i386 folder for current R).
> >>
> >> From: Jeroen Ooms  Sent: Thursday, 9
> >> April 2020 21:32 To: Bravington, Mark (Data61, Hobart)
> >> Cc: R-Devel-2 Subject: Re: [Rd] missing binaries in
> >> R-devel windows snapshot 78175
> >>
> >>
> >> > I just tested this and as far as I can tell there are
> >> no missing > files. Are you sure there isn't a local
> >> problem with your system > permissions or antivirus that
> >> is removing the files?
> >>
> >> You're quite right--- my apologies. I've been Cylanced
> >> (without it having the politeness to actually tell me,
> >> g). I tried again with the 78172 build that you used,
> >> and the two dot-exe files are there immediately after
> >> installation--- but 10 minutes later they're gone. One
> >> question below about the md5 stuff, though:
> >>
> >> > Try to verify the md5 of the installer; some enterprise
> >> firewalls are > tampering with downloads:
> >>
> >> >
> >> 
> openssl::md5(url('https://cran.r-project.org/bin/windows/base/R-devel-win.exe'))
> >> >
> >> 
> readLines('https://cran.r-project.org/bin/windows/base/md5sum.txt.R-devel')
> >>
> >> Thanks for the tip. I did that, and both give the same
> >> output. But:
> >>
> >> - If I run 'md5sum' on my local copy of the 78182
> >> installer (ie the file I've just downloaded) I again get
> >> the same signature as per your two lines. (It's not the
> >> firewall, it's Cylance.)
> >>
> >> - Yet if I run 'openssl::md5( )'
> >> I get a different signature!
>
> > I think you're calculating the md5 of the filename, not
> > the file. You need to use:
>
> >   openssl::md5(file("R-devel-win.exe"))
>
> or use base R'stools::md5sum()
>
> md5sum() has existed forever, but needs you to do the
> download.file() step  which is indirectly happening in
> openssl::md5() also.

Not entirely the same way though! The openssl implementation of md5,
sha256, etc calculate the hash directly from the connection stream,
without ever storing the entire file on disk or in memory. It
efficiently hashes arbitrary large data from any readable R connection
(url, file, pipe, etc) using a fixed amount of memory. This is
important for researchers that calculate hashes for very large data
files, where copying the entire data to disk or memory is impossible.

However, in this case it's just a small file, so tools::md5sum() is
simpler indeed :)

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Rtools and R 4.0.0?

[Jeroen Ooms]

Hevré told me that people from Bioconductor and possibly others are
keeping an eye on this thread so it would be good to post a note here
too.

As of version 4.0.0, the official R for Windows and binary packages
provided via CRAN are built with gcc-8.3.0 from rtools40, as shown in
the CRAN check table:
https://cran.r-project.org/web/checks/check_flavors.html

Instructions and installers for rtools40 are available at the usual
location: https://cran.r-project.org/bin/windows/Rtools/ . Suggestions
for improving the documentation can be posted here:
https://github.com/r-windows/docs








On Tue, Apr 7, 2020 at 6:07 PM Kevin Ushey  wrote:
>
> That's great to see, although I suspect it's still a speculative
> change and could be backed out if any non-trivial issues were
> encountered.
>
> Regardless, I would like to thank R core, CRAN, and Jeroen for all of
> the time that has gone into creating and validating this new
> toolchain. This is arduous work at an especially arduous time, so I'd
> like to voice my appreciation for all the time and energy they have
> spent on making this possible.
>
> Best,
> Kevin
>
> On Tue, Apr 7, 2020 at 7:47 AM Dirk Eddelbuettel  wrote:
> >
> >
> > There appears to have been some progress on this matter:
> >
> > -Note that @command{g++} 4.9.x (as used for @R{} on Windows up to 3.6.x)
> > +Note that @command{g++} 4.9.x (as used on Windows prior to @R{} 4.0.0)
> >
> > See SVN commit r78169 titled 'anticipate change in Windows toolchain', or 
> > the
> > mirrored git commit at
> > https://github.com/wch/r-source/commit/bd674e2b76b2384169424e3d899fbfb5ac174978
> >
> > Dirk
> >
> > --
> > http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] "not a valid win32 application" with rtools40-x86_65.exe on Windows 10

[Jeroen Ooms]

On Thu, Apr 30, 2020 at 6:38 AM Spencer Graves
 wrote:
>
> Hello, All:
>
>
>"00install.out" from "R CMD check Ecfun_0.2-4.tar.gz" includes:
>
>
> Error:  package or namespace load failed for 'Ecfun':
>   .onLoad failed in loadNamespace() for 'rJava', details
>call: inDL(x, as.logical(local), as.logical(now), ...)
>error:  unable to load shared object 'c:/Program
> Files/R/R-4.0.0/library/rJava/libs/i386/rJava.dll':
>LoadLibrary failure: ^1 is not a valid win32 application
>

This is an error in loading the rJava package, so it is not related to
rtools40, and probably inappropriate for this mailing list.

As Simon suggested, you may have to install the 32-bit Java JDK. See
also this faq: 
https://github.com/r-windows/docs/blob/master/faq.md#how-to-install-rjava-on-windows

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Tcl.Tk Tktable package missing from R 4.0.0 on Windows

[Jeroen Ooms]

On Sun, May 3, 2020 at 6:15 PM Fox, John  wrote:
>
> Dear R-devel list members,
>
> The Tktable package for Tcl/Tk is apparently missing from the Windows 
> distribution of R 4.0.0. I (actually a user of the Rcmdr package) discovered 
> this when trying to use the new-data-set dialog in the Rcmdr package, 
> producing the error, "Tcl package 'Tktable' must be installed first."
>
> I believe the Tktable has been part of the R distribution for Windows since R 
> version 2.9.0, and is still present in the macOS distribution of R 4.0.0.
>
> I apologize for not discovering this problem prior to the release of R 4.0.0. 
> I did test the Rcmdr package under R 4.0.0 on both Windows and macOS, but not 
> every menu item and dialog on each platform.
>
> Does anyone have more information about this problem?

Sorry this was my mistake. I somehow missed tktable when updating the
tcltk bundle for rtools40. This didn't show up in any package check
either.

I've added it back now, it should work again in today's builds r-devel
and r-patched. Thanks for catching this.

Jeroen

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Build failing on win64

[Jeroen Ooms]

As of commit 78536 earlier this morning the build is failing on
windows 64, see https://r-devel.github.io

I cannot immediately spot what is the problem. The build fails with:

installing 'sysdata.rda'
  make[3]: *** [../../../share/make/basepkg.mk:151: sysdata] Error 127
  make[2]: *** [Makefile.win:22: all] Error 2
  make[1]: *** [Makefile.win:32: R] Error 1

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Disable interaction in R to use with Rserve

[Jeroen Ooms]

Is there a way to disable interactive behavior in R (?interactive), but
without it affecting the default error handling and reporting? This would be
needed to use Rserve without any interruption. The problem is that the
Rserve expects every eval either to return or return an error. The
connection gets stuck whenever R wants to display some kind of prompt.

For example, when the user runs install.packages for the first time and
doesn't supply the repos argument, R displays a list of CRAN mirrors. When
this happens, Rserve does not return; it just waits there, and the Rserv
connection becomes unattainable.

As a possible solution, we tried to disable R interactive mode (see
?interactive). The CRAN package 'interactivity' provides a function to
toggle C level global variable R_Interactive. However, setting interactive
mode to false, also results in that R exits the R session on every error.
This is not what we want, we would like to keep the R sesion alive and leave
the R error handling unaffected, and only suppress any prompts or dialogs
that prevent the call from returning.

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] dependencies on system packages

[Jeroen Ooms]

Many R packages depend on some unix libraries that are not part of most
default installations. I often spend a significant amount of time figuring
out where to get the appropriate libraries for compiling these packages,
after they give some vague error of something missing. I was wondering why
there is no formal system of specifying non-R dependencies in the
DESCRIPTION file. If this would be the case, then during the installation of
an R package, the user could be prompted to install required system packages
(if they have appropriate privileges).

So for example:

Package: XML
Version: 3.2-0
Depends: R (>= 1.2.0), methods, utils
Depends-debian: libxml2-dev
Depends-ubuntu: libxml2-dev
Depends-redhat: libxml2-devel
Depends-suse: libxml2-devel
etc. 

This might make life for many people just a little easier. If they are root
and the package is in their system repositories, than it will install
automatically. If not, at least they know for which package to look, or
request their sys admin to install.
-- 
View this message in context: 
http://r.789695.n4.nabble.com/dependencies-on-system-packages-tp3259395p3259395.html
Sent from the R devel mailing list archive at Nabble.com.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] call / do.call expression too big

[Jeroen Ooms]

I am designing a remote procedure protocol for R, which basically performs a
do.call on data and arguments supplied by a client. However, I am
experiencing unfortunate behavior of the do.call function. The function
do.call seems to serialize all objects in the args list, resulting in an
enormous expression, that is then in some way or another returned by many
functions. A short example:

args <- list(data=cars, formula=dist~speed);
do.call("lm",args);

One can see that for this small example, the returned 'call' attribute is
already huge. Another example:

eval(call("cor.test", x=rnorm(100), y=rnorm(100)))

I completely understand why this happens, yet I was wondering if there is an
alternative to call/do.call that constructs the call in an different way so
that the actual call object is somewhat more compact and does not contain
all the data that was involved in the function. For example one that
attaches the args list in an environment and constructs a call that refers
to these objects or something similar.


--
View this message in context: 
http://r.789695.n4.nabble.com/call-do-call-expression-too-big-tp3574335p3574335.html
Sent from the R devel mailing list archive at Nabble.com.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] call / do.call expression too big

[Jeroen Ooms]

>
> argn <- lapply(names(args), as.name)
> names(argn) <- names(args)
>
> call <- as.call(c(list(as.name("lm")), argn))
> eval(call, args)
>

Great, almost! Is there any way I can explicitly specify the package of the
function in this way? I tried replacing "lm" with "stats::lm" but that did
not work.

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [linux] connection never times out

[Jeroen Ooms]

> Can you please verify the behaviour is still the same in a recent R-devel or
> at least R-2.13.1? And that there was no other already answered request on
> R-help or R-devel re. timeouts?

The code below is R 2.13.1. It shows that the timeout time is more
than 3 minutes, although it was set to 5 seconds.

> options(timeout=5)
> system.time(download.file("http://123.123.123.123";, dest=tempfile()))
trying URL 'http://123.123.123.123'
Error in download.file("http://123.123.123.123";, dest = tempfile()) :
  cannot open URL 'http://123.123.123.123'
In addition: Warning message:
In download.file("http://123.123.123.123";, dest = tempfile()) :
  unable to connect to '123.123.123.123' on port 80.
Timing stopped at: 0 0 189.375
> sessionInfo()
R version 2.13.1 (2011-07-08)
Platform: i686-pc-linux-gnu (32-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=C  LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
 [9] LC_ADDRESS=C   LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Randomness not due to seed

[Jeroen Ooms]

>> I think Bill Dunlap's answer addressed it:  the claim appears to be false.

Here is another example where there is randomness that is not due to
the seed. On the same machine, the same R binary, but through another
interface. First directly in the shell:

> sessionInfo()
R version 2.13.1 (2011-07-08)
Platform: i686-pc-linux-gnu (32-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=C  LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
 [9] LC_ADDRESS=C   LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

> set.seed(123)
> print(coef(lm(dist~speed, data=cars)),digits=22)
  (Intercept) speed
-17.579094890510951643137   3.932408759124087715975



# And this is through eclipse (java)

> sessionInfo()
R version 2.13.1 (2011-07-08)
Platform: i686-pc-linux-gnu (32-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8  LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8   LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8   LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8  LC_NAME=en_US.UTF-8
 [9] LC_ADDRESS=en_US.UTF-8LC_TELEPHONE=en_US.UTF-8
[11] LC_MEASUREMENT=en_US.UTF-8LC_IDENTIFICATION=en_US.UTF-8

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

other attached packages:
[1] rj_0.5.2-1

loaded via a namespace (and not attached):
[1] rJava_0.9-1  tools_2.13.1

> set.seed(123)
> print(coef(lm(dist~speed, data=cars)),digits=22)
 (Intercept)speed
-17.57909489051087703615   3.93240875912408460735

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] RApache error on RHEL/CentOS

[Jeroen Ooms]

I am running RApache (www.rapache.net) on CentOS 5.6 using the R
binary from EPEL. After upgrading to R from 2.12 to 2.13 I am getting
the following error when starting Apache:

Cannot load /usr/lib/httpd/modules/mod_R.so into server: libgomp.so.1:
shared object cannot be dlopen()ed

I tried rebuilding RApache, but the error remains. Installed are:

R-devel.i386 2.13.0
libgomp.i386 4.1.1-52.el5.2
httpd (apache) 2.2.3
rapache 1.1.4

Has anything changed in R 2.13 that might cause this issue?

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] [foreign] read.spss 'measure' attribute

[Jeroen Ooms]

In SPSS compatible datafiles, every variable has a property called
'Measure'. This attribute can be observed as the last column in 'variable
view mode' when the dataset is opened in e.g. GNU PSPP. The property is not
very important, but is used by many researchers to distinguish between
'Ordinal' and 'Nominal' variables; respectively ordered and unordered
factors in R.

The measure property does not seem to be extracted by the foreign::read.spss
function. It would be nice if there would be a way to read this information,
and add is as an attribute to every variable.

A very useful feature (at least for me :-) would be to include an option in
read.spss that variables which are currently converted to an R factor, will
be converted to an ordered.factor when Measure equals 'Ordinal'. Or in
pseudocode:

After reading in the spss dataset and converting it to a data.frame:
for var in data.frame:
if ( class ( var ) == "factor" && Measure == "Ordinal" ) {
class ( var ) <- c( "ordered", "factor" )
}

Here a datafile to play with:

download.file("http://www.stat.ucla.edu/~jeroen/files/1991GS.sav";,
"1991GS.sav");
mydata <- read.spss("1991GS.sav", to.data.frame=T);
sapply(mydata, is.ordered);

Thanks.

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Overwriting imported function in another package

[Jeroen Ooms]

I am running into a limitation of the grid::grid.newpage function, for
which I would like to overwrite this function with a slightly modified
one. Hopefully this is a temporary working solution until the package
gets updated. I found a way to overwrite the function in the
package:grid namespace. However, lattice imports grid rather than
depending on it. So I need a way to overwrite this imported version as
well. The code below shows the fix which works for ggplot (because it
depends on grid), but it doesn't work for lattice, because it imports
grid. Is there any way to overwrite grid.newpage for all
instantiations of it?

#packages
library(grid);
library(lattice);
library(ggplot2);

#create the modified function.
hookfun <- deparse(body(plot.new))[1:6]
oldfun <- deparse(body(grid::grid.newpage))[-1];
newfun <- grid::grid.newpage;
body(newfun) <- parse(text=c(hookfun, oldfun));

#overwrite it in the package
unlockBinding("grid.newpage", as.environment("package:grid"))
assign("grid.newpage", newfun, pos="package:grid")

#this seems ok:
get('grid.newpage', as.environment("package:grid"));
get('grid.newpage', as.environment("package:lattice"));

#but this is still the old one
get('grid.newpage', environment(histogram));

#test if it worked:
setHook("before.plot.new", function() {message("Yay! A new plot.");});
qplot(rnorm(100)); #it worked for ggplot2
histogram(rnorm(100)); #didn't work for lattice

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Overwriting imported function in another package

[Jeroen Ooms]

> Yes, modify the source and recompile R.

That is what I am doing now, but can't expect that everyone who uses
my code is willing to recompile R from src...

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Compiling 64bit static library for Windows (Rtools33, MSYS2, cross-compile on linux)

[Jeroen Ooms]

Hi Sue,

I maintain a few static libs for R packages on
https://github.com/rwinlib. If your R packages are open source I can
try to add the libraries you need. Answers inline:

>Can Rtools33 be used to compile static libraries as a separate step
>using -m64 flag?

Yes, but usually you'll need msys to as well to run configure, etc.

>MSYS2 is very convenient for compiling many of these libraries. Can a
>static library compiled with x86_64-w64-mingw32 (gcc 4.9.2) be used with
>Rtools33 (v 4.6.2)?

Sometimes. It will not work for c++ libraries because the gcc build
included with msys2 uses a different exception and threading models.
For C libraries, sometimes building in msys2 leads the static library
to depend on certain system libraries that are included with msys2 but
not with rtools.

>Can a static library compiled using x86_64-w64-mingw32-gcc
>cross-compiler (v 4.6.2) on linux be used with Rtools33?

If the library is built properly, this is usually not a problem; most
builds do not depend on a particular gcc version. For example static
and dynamic libraries for libcurl and dependencies (openssl, libssh2,
libz, etc) are available via http://curl.haxx.se/gknw.net/7.40.0/ and
in my experience these can be linked with any somewhat recent version
of gcc.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] NEWS.md support on CRAN

[Jeroen Ooms]

On Tue, Jun 2, 2015 at 8:42 PM, Duncan Murdoch  wrote:
> Several of us have been talking about this, and have more or less decided 
> against pandoc for the within-R version.  It is too hard to build, or to 
> include with R, on weird platforms. Something like commonmark is more likely. 
> We'd also like to separate the parse and render steps.

The commonmark::markdown_xml function gives the parse tree in xml
format. From there we could use something like xpath to query the
content for individual news items and such. I think this is a more
practical way of exposing the syntax tree than deeply nested lists or
an external pointer to the internal C handle.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] CRAN testing

[Jeroen Ooms]

In addition to what has been said already, you should check C code for
-pedantic compiler warnings (in particular GNU extensions) and verify
that the package builds with parallel make (e.g. MAKE=make -j8).




On Fri, Jun 12, 2015 at 7:24 AM, Mick Jordan  wrote:
>
> Is the mechanism by which packages are tested on CRAN described
> anywhere? Is it by any chance written in R? The FastR
>  team is interested in
> running a virtual CRAN where we can test all the CRAN packages on FastR.
>
> Thanks
> Mick Jordan
>
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenMP problem with 64-bit Rtools

[Jeroen Ooms]

Hi Radford

On Fri, Aug 21, 2015 at 8:38 PM, Radford Neal  wrote:
>
> I've been getting pqR to work on windows systems, and in the process
> have discovered various problems with R core versions of R and with
> Rtools.


We happen to be working on a new version of the windows tool chain,
perhaps you are interested to test if problems still exist in the
latest versions. Find a copy of gcc-4.9.2 as well as an installer with
the latest r-devel built with this compiler here:
http://www.stat.ucla.edu/~jeroen/mingw-w64/. Note that this R
installer contains a copy of the tool chain (hence requires no
seperate 'rtools' installation).

> The reason for this appears to be that the omp.h file included is
> appropriate only for the 32-bit version of the openMP library.  As
> far as I can see, there is no 64-bit version of this include file
> in the stuff installed with Rtools.


This particular behavior exists in all builds of mingw-w64, not rtools
per se. We might get a better answer in the mingw-w64-public mailing
list.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Issues with libcurl + HTTP status codes (eg. 403, 404)

[Jeroen Ooms]

On Tue, Aug 25, 2015 at 10:33 PM, Martin Morgan  wrote:
>
> actually I don't know that it does -- it addresses the symptom but I think 
> there should be an error from libcurl on the 403 / 404 rather than from 
> read.dcf on error page...

Indeed, the only correct behavior is to turn the protocol error code
into an R exception. When the server returns a status code >= 400, it
indicates that the request was unsuccessful and the response body does
not contain the content the client had requested, but should instead
be interpreted as an error message/page. Ignoring this fact and
proceeding with parsing the body as usual is incorrect and leads to
all kind of strange errors downstream.

The other download methods did this correctly, it is unclear why the
current implementation of the "libcurl" method does not. Not only does
it lead to hard to interpret downstream parsing errors, it also makes
the behavior of R ambiguous as it is dependent on which download
method is in use. It is certainly not a limitation of the libcurl
library: the 'curl' package has alternative implementations of url()
and download.file() which exercise the correct behavior.

I can only speculate, but if the motivation is to explicitly support
retrieval of error pages, perhaps the download.file() and url()
functions can gain an argument 'stop_on_error' or something similar
which give the user an option to ignore server errors. However this
behavior should certainly not be the default. When a function or
script contains a line like this:

  download.file("https://someserver.com/mydata.csv";, "mydata.csv")

Then in the next line of code we must be able to expect that the file
"mydata.csv" we have downloaded to our disk is in fact the file
"mydata.csv" that was requested from the server. An implementation
that instead saves an error page (likely html content) to the
"mydata.csv" file is simply incorrect and will lead to obvious
problems, even with a warning.


[1] https://www.opencpu.org/posts/cran-https/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Issues with libcurl + HTTP status codes (eg. 403, 404)

[Jeroen Ooms]

On Thu, Aug 27, 2015 at 5:16 PM, Martin Maechler
 wrote:
> Probably I'm confused now...
> Both R-patched and R-devel give an error (after a *long* wait!)
> for
>download.file("https://someserver.com/mydata.csv";, "mydata.csv")
>
> So that problem is I think  solved now.

I'm sorry for the confusion, this was a hypothetical example.
Connection failures are different from http status errors. Below some
real examples of servers returning http errors. For each example the
"internal" method correctly raises an R error, whereas the "libcurl"
method does not.

# File not found (404)
download.file("http://httpbin.org/data.csv";, "data.csv", method = "internal")
download.file("http://httpbin.org/data.csv";, "data.csv", method = "libcurl")
readLines(url("http://httpbin.org/data.csv";, method = "internal"))
readLines(url("http://httpbin.org/data.csv";, method = "libcurl"))

# Unauthorized (401)
download.file("https://httpbin.org/basic-auth/user/passwd";,
"data.csv", method = "internal")
download.file("https://httpbin.org/basic-auth/user/passwd";,
"data.csv", method = "libcurl")
readLines(url("https://httpbin.org/basic-auth/user/passwd";, method =
"internal"))
readLines(url("https://httpbin.org/basic-auth/user/passwd";, method = "libcurl"))

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Default location where packages are stored under Windows

[Jeroen Ooms]

On Sat, Sep 5, 2015 at 2:05 PM, Tom Wenseleers
 wrote:
> I was just wondering if the default directory to store packages on Windows 
> platforms could therefore not better be made the personal library folder, 
> Users/Username/Documents/R/win-library/3.XX/


I think this is already the case? The first time you try to install a
package as non-admin, you will be prompted with a message that the
global library is not writable and if you want to create a user
library. Once the user library exists, R will make that the default by
putting it on the head of .libPaths() at the start of every R session.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Optimization bug when byte compiling with gcc 5.2.0 on windows

[Jeroen Ooms]

When building R-devel with gcc 5.2.0 (mingw-w64 v4) on Windows, make
check fails reg-tests-1b.R at the following check:

  x <- c(1:2, NA)
  sx <- sd(x)
  !is.nan(sx)

Here 'sx' should be 'NA' but it is 'NaN'. It turns out this problem
only appears when the function is byte compiled with optimization
level 3:

  mysd <- function (x, na.rm = FALSE)
sqrt(var(if (is.vector(x)) x else as.double(x), na.rm = na.rm))

  mysd(x)
  # [1] NA
  compiler::cmpfun(mysd, list(optimize = 1L))(x)
  # [1] NA
  compiler::cmpfun(mysd, list(optimize = 2L))(x)
  # [1] NA
  compiler::cmpfun(mysd, list(optimize = 3L))(x)
  # [1] NaN

The problem does not appear with gcc 5.2.0 on Debian, and also not
with gcc 4.9.3 on Windows. Where do I start debugging this? The
disassembled output from the compiled functions is here:
https://gist.github.com/jeroenooms/3206945a6db6680a9c5c

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Optimization bug when byte compiling with gcc 5.2.0 on windows

[Jeroen Ooms]

On Mon, Sep 14, 2015 at 4:16 PM, Duncan Murdoch
 wrote:
> I think the reason for the earlier restriction is the assumption that
> eventually gcc will be fixed and this workaround won't be necessary, but
> apparently 5.2.0 still has the same problem.

I tested this hypothesis by replacing '__GNUC__ <= 4' with '__GNUC__
<= 5' and rebuilding R, but this introduces a whole lot of problems
that were not there before. In particular R crashes when using the
graphics device, which it did not without this fix.

I first thought this was an unrelated random crash, but 'make check'
consistently fails in base-Ex.R at the first invocation of boxplot().
Same for 'check recommended', each package fails when making a plot.
None of this happens with the original source code, where everything
works fine, besides the optimized version of sqrt(NA) giving NaN. Did
something perhaps change in gcc 5 for the ISNAN macro that makes it
unsuitable to be used in this way?

FWIW it looks like the original fix was introduced by Luke about a
year ago in r66652:
https://github.com/wch/r-source/commit/e9aaf8fdeddf27c2a9078cd214a41475c8ff6f40

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Optimization bug when byte compiling with gcc 5.2.0 on windows

[Jeroen Ooms]

On Mon, Sep 14, 2015 at 8:49 PM, Jeroen Ooms  wrote:

> I tested this hypothesis by replacing '__GNUC__ <= 4' with '__GNUC__
> <= 5' and rebuilding R, but this introduces a whole lot of problems
> that were not there before. In particular R crashes when using the
> graphics device, which it did not without this fix.

I found that the problem goes away when we use isnanl() instead of
isnan(). This is very similar to the problem we have for R_pow() where
we need to use powl() instead of pow() for recent versions mingw-w64
(this still needs to be patched in r-devel).

So the full solution is:

#if (defined(_WIN32) || defined(_WIN64)) && defined(__GNUC__)
# define R_sqrt(x) (isnanl(x) ? x : sqrt(x))
#else
# define R_sqrt sqrt
#endif

Below the relevant mingw-w64 sources:

https://github.com/Alexpux/mingw-w64/blob/master/mingw-w64-crt/math/isnan.c
https://github.com/Alexpux/mingw-w64/blob/master/mingw-w64-crt/math/isnanf.c
https://github.com/Alexpux/mingw-w64/blob/master/mingw-w64-crt/math/isnanl.c

I think mingw-w64 takes powl from glibc.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R-devel_2015-09-14 throws an error in reg-packages test

[Jeroen Ooms]

On Wed, Sep 16, 2015 at 9:32 PM, Avraham Adler  wrote:

> When running make check, I get the following error:
>
> running code in 'reg-packages.R' ...make[3]: *** [reg-packages.Rout] Error 1
> make[2]: *** [test-Reg] Error 2
> make[1]: *** [test-all-basics] Error 1
> make: *** [check] Error 2
>
> Looking at reg-packages.Rout.fail shows

The r-devel commit log [1] shows that Martin has been working on this
test over the past few days. Sometimes work-in-progress causes the
daily svn snapshot to be broken; typically it gets fixed the next day
or so. If you don't want to wait, try building
R-devel_2015-09-10.tar.gz that one works.

[1] https://github.com/wch/r-source/commits/trunk

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] MiKTeX's withdrawal of texi2dvi.exe

[Jeroen Ooms]

On Sat, Oct 10, 2015 at 8:02 AM, Prof Brian Ripley
 wrote:
>
> It was used to make manuals when building R from sources.  This has been 
> worked around in R-patched/R-devel.  If you need to build an earlier version 
> of R, edit doc/manuals/Makefile.win ...

Thank you for the fix.

FWIW, texi2dvi.exe was actually removed quite a while ago (1 year?). A
copy of an older miktex build of the current miktex branch that still
includes texi2dvi is available
here:http://www.stat.ucla.edu/~jeroen/mingw-w64 . Perhaps it is worth
archiving this somewhere to be able to re-build older versions of R on
Windows in the future.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Package integrity check via SHA256 or OpenPGP possible?

[Jeroen Ooms]

On Thu, Oct 15, 2015 at 9:11 AM, Philip Gillißen  wrote:
> I'm using R in a corporate environment...

That's irrelevant.

> is possible to force R to verify integrity via SHA256 or even OpenPGP 
> signatures? If not are there any plans to support better hashes than MD5? As 
> the source code looks, an extension to support other (optional) hash values 
> would be quite easy.

A hash is not the same thing as a signature. If you need an
introduction to these topics I recommend www.crypto101.io.

Adding sha256 support would indeed be easy but we wouldn't gain much.
Coincidental md5 collisions are very unlikely. If you are considering
deliberate attacks on the network, a hash function is not going to
help as the attacker can just recompute the hash along with the
tampered package.

Also note that if you are using an https mirror, the connection
between you and the mirror should be secure. But of course it is still
possible that a mirror server itself gets hacked.

Now, signing packages with a private key would be useful against such
attacks, but require quite a bit of work. If we would follow the
Debian model, it would require that:

 - Each of the PACKAGES files in the repository gains a secure hash
(e.g. sha256) for each file in the directory.
 - A "Release" file [1] is added to the repository that includes a
secure hash of each of the PACKAGES files.
 - A "Release.gpg" file [2] or equivalent is added which contains a
signature of the "Release" file.
 - Each time "PACKAGES" and hence "Release" files are updated, the
repository maintainers need to update "Release.gpg" by signing
"Release" with their private key.
 - R installations would have to ship with the corresponding public
key in order to verify the signature in Release.gpg before installing
packages.
 - All of this would have to be implemented using a crypto library
capable of public key signatures.
 - A policy is designed for rotating keys when they expire, or between
R releases, or when one gets compromised, etc.

I have actually been looking into this over the past few months. On
Linux it would be doable with gpg, but unfortunately gpg does not work
very well on OSX and Windows, which make it impractical for R.

There are a few alternatives that are more portable. The PKI package
uses openssl to do RSA signatures, which could work. The new 'sodium'
package is probably the most advanced thing currently available. It
uses the ed25519 public-key signature system with a very simple API.
See the manual page for the 'sig_sign' and 'sig_verify' functions.

Either way, all this would require a substantial amount of work and
additional ongoing maintenance in both R and CRAN, which is unlikely
to happen given the limited resources.


[1] http://http.us.debian.org/debian/dists/jessie/Release
[2] http://http.us.debian.org/debian/dists/jessie/Release.gpg

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R 3.2.2 - make check and install package hang

[Jeroen Ooms]

On Sat, Oct 17, 2015 at 9:04 PM, McCully, Dwayne (NIH/NIAMS) [C]
 wrote:
> After trying several ways to compile R 3.2.2 without luck, I’m reaching out 
> for help.

If you are new to R you probably want to use the precompiled binaries.
On CentOS/RHEL these are available through the EPEL repositories (see
http://bit.ly/1xVxGlD for installation instructions)

Once EPEL is installed you can simply do 'yum install R-devel' as you
would do on Fedora.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Custom C finalizers for .Call

[Jeroen Ooms]

WRE explains that R_alloc() can be used to allocate memory which
automatically gets released by R at the end of a .C, .Call or
.External, even in the case of an error or interruption. This is a
really great feature to prevent memory leaks. I was wondering if there
is a way to extend this mechanism to allow for automatically running
UNPROTECT and custom finalizers at the end of a .Call as well.

Currently it is all to easy for package authors to introduce a memory
leak or stack imbalance by calling Rf_error() or
R_CheckUserInterrupt() in a way that skips over the usual cleanup
steps. This holds especially for packages interfacing C libraries
(libcurl, libxml2, openssl, etc) which require xx_new() and xx_free()
functions to allocate/free various types of objects, handles and
contexts. Therefore we cannot use R_alloc() and we need to manually
clean up when returning, which is tricky for irregular exits.

Moreover package authors might benefit from an alternative of
allocVector() which automatically protects the SEXP until the .Call is
done. Perhaps I don't fully appreciate the complexity of the garbage
collector, but one could imagine a variant of PROTECT() which
automatically keeps a counter 'n' for the number of allocated objects
and makes R run UNPROTECT(n) when .Call exists, along with releasing
the R_alloc() memory. Yes, there are cases where it is useful to have
manual control over what can be collected earlier during the .Call
procedure, but these are rare. A lot of C code in packages might
become safer and cleaner if authors would have an option to let this
be automated.

The most general feature would a hook for adding custom C functions to
the .Call exit, similar to on.exit() in R:

  xmlNodePtr *node =  xmlNewNode(...);
  Rf_on_exit(xmlFreeNode, node);
  EVP_PKEY_CTX *ctx = EVP_PKEY_CTX_new(...);
  Rf_on_exit(EVP_PKEY_CTX_free, ctx);
  SEXP out = PROTECT(allocVector(...));
  Rf_on_exit(UNPROTECT, 1);

I don't know R's internals well enough to estimate if something like
this would be possible. I did put together a simple C example of a
linked list with object pointers and their corresponding free
functions, which can easily be free'd with a single call:
http://git.io/vBqRA . So basically what is mostly missing at this
point is a way to trigger this at the end of the .Call in a way that
works for regular returns, errors and interruptions...

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] How do I reliably and efficiently hash a function?

[Jeroen Ooms]

On Fri, Dec 11, 2015 at 12:49 AM, Konrad Rudolph
 wrote:
>
> On the chance that I’m trying to solve the wrong Y to an X/Y problem,
> the full context to the above problem is explained in [1]. In a
> nutshell, I am hooking a new environment into a function’s parent.env
> chain, by re-assigning the function’s `parent.env` (naughty, I know).
> This is done so that the function `f` finds objects defined inside
> that environment without having to attach it globally.

Not sure if this is helpful, but you can implement this more naturally
using closures without hacking on environments. As I understand it,
your functions have some shared state, and some private. So each
function needs a private parent env, which all share a common
grand-parent that holds your shared objects. Maybe this example helps:

new_closure <- (function(shared = 0){
  function(name, priv = 0){
function(){
  priv <<- priv +1
  shared <<- shared +1
  print(sprintf("Total:%d; %s:%d", shared, name, priv))
}
  }
})()

fun1 <- new_closure("erik")
fun2 <- new_closure("anna")

fun1()
fun1()
fun1()
fun2()
fun1()
fun1()
fun2()

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Crash when reusing libcurl connection

[Jeroen Ooms]

Reusing a libcurl based url() connection can lead to a crash:

  con <- url("http://httpbin.org/get";, method = "libcurl")
  readLines(con)
  readLines(con)
  readLines(con)

The problem does not appear for other url() implementations:

  con <- url("http://httpbin.org/get";, method = "internal")
  readLines(con)
  readLines(con)

  con <- curl::curl("http://httpbin.org/get";)
  readLines(con)
  readLines(con)

>From the internet/libcurl.c source code it looks like the free() calls
should be moved out of Curl_close() and into the destroy callback
(which is currently undefined?). Comments from Connections.h suggest
that the close callback should reset, but not destroy the connection
so that it can be recycled:

  void (*close)(struct Rconn *); /* routine closing after auto open */
  void (*destroy)(struct Rconn *); /* when closing connection */

The 'curl' package has a working example how to achieve this with libcurl.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] default destfile in download.file()

[Jeroen Ooms]

A nice default value for the `destfile` argument in download.file() would
be `basename(url)` i.e. the name of the downloaded file. This would
correspond to default behavior in many other web/ftp clients and makes code
slightly more concise:

  download.file("https://svn.r-project.org/R/trunk/doc/CRAN_mirrors.csv";)
  mirrors <- read.csv("CRAN_mirrors.csv")

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] checking https certificates

[Jeroen Ooms]

It looks like the https certificate on r-project.org is currently
expired. I am sure sysadmin in Vienna is on it, but perhaps we can
prevent this from happening again by adding a check in R or the mirror
report [1] to warn about mirrors with https certs that are about to
expire:

  library(openssl)
  certs <- download_ssl_cert("r-project.org")
  cert_verify(certs)
  # Error in cert_verify_cert(cert[[1]], cert[-1], root) :
  #   Certificate validation failed: certificate has expired
  as.list(certs[[1]])$validity
  # [1] "Mar 10 17:14:13 2014 GMT" "Mar 10 17:14:13 2016 GMT"

PS: the cert that is deployed on r-forge.r-project.org is valid for
*.r-project.org and is valid until 2018 so that should make an easy
fix:

  cert <- download_ssl_cert("r-forge.r-project.org")
  cert_verify(cert)
  as.list(cert[[1]])


[1] https://cran.rstudio.com/mirmon_report.html

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Optimization bug when byte compiling with gcc 5.3.0 on windows

[Jeroen Ooms]

On Sun, Apr 3, 2016 at 6:44 PM, Ray Donnelly  wrote:
> I've started to look into building R for Windows using MSYS2 as both the
> build environment and tools + libraries provider (where possible).

Thanks for your reply, and for the patches.

Last time I had a look at this (a few months ago) another problem was
that mingw-w64 v4 was giving different numeric output for some of the
tests in r-base. If I recall correctly, some eigen vectors had their
direction flipped (negative values became positive and vice versa).
Did you notice anything of this kind when running 'make check' and
'make check recommended' ? It is important to us that numeric results
are reproducible between versions of R.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R process killed when allocating too large matrix (Mac OS X)

[Jeroen Ooms]

On 05/05/2016 10:11, Uwe Ligges wrote:
> Actually this also happens under Linux and I had my R processes killed
> more than once (and much worse also other processes so that we had to
> reboot a server, essentially).

I found that setting RLIMIT_AS [1] works very well on Linux. But this
requires that you cap memory to some fixed value.

> library(RAppArmor)
> rlimit_as(1e9)
> rnorm(1e9)
Error: cannot allocate vector of size 7.5 Gb

The RAppArmor package has many other utilities to protect your server
such from a mis-behaving process such as limiting cpu time
(RLIMIT_CPU), fork bombs (RLIMIT_NPROC) and file sizes (RLIMIT_FSIZE).

[1] http://linux.die.net/man/2/getrlimit

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] unloadNamespace problem in 3.3

[Jeroen Ooms]

The following used to work in R 3.2.5 but not in 3.3.0:

  library(MASS)
  ns <-.getNamespace("MASS")
  unloadNamespace(ns)

Calling unloadNamespace("MASS") directly still works.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R process killed when allocating too large matrix (Mac OS X)

[Jeroen Ooms]

On Thu, May 12, 2016 at 9:51 AM, Martin Maechler
 wrote:
> My conclusion was I could not use the RAppArmor package.
>
> (But that's wrong: For the  rlimit*()  functions below, one do
>  *NOT* need an AppArmor-enabled version of Linux !)

Yes, it is a relatively recent (unadvertised) feature that the package
now builds on linux systems without libapparmor. I agree this names
the package name confusing. I'll make at least that warning more
informative.

Some background: When I started the package (5 years ago) I expected
that soon all linux distributions would have the apparmor module which
has been in the kernel since 2.6.36. However Redhat is explicitly
disabling it because they are pushing a competing MAC system (SELinux)
which they develop together with the NSA, and they really want you to
use this instead (and only this!).

> I gather that all of these are *not* Apparmor related... so could/should 
> maybe rather migrate into a lightweight package not mentioning AppArmor ?

I agree, it has been on the to do list for a while; Kirill and me were
talking yesterday about what would be the best route to take:

 - A small package with only the rlimit bindings
 - or: A 'linux' package with bindings to anything in the kernel,
including rlimit, but possibly other system tools.
 - or: A package targeting POSIX/unix with standard functionality that
is also available on OSX/BSD.

>From my experience, windows is pretty useless for this kind of stuff.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Suggestion: mkString(NULL) should be NA

[Jeroen Ooms]

I would like to propose that Rf_mkString(NULL) and Rf_mkChar(NULL)
return NA rather than segfault.

Case: the mkString() and mkChar() functions are convenient to wrap
strings returned by e.g. external C libraries into an R vector.
However sometimes a library returns NULL instead of a string when the
result is unavailable. In some C libraries this can happen
unexpectedly or is even undocumented.

A good R package author always checks results for a null pointer, and
deals with it accordingly. But sometimes we make assumptions. There
was an example in the 'curl' package where a documented version string
was suddenly NULL if libcurl was built with some unusual
configuration. These problems are hard to catch and I don't see any
benefit of segfaulting for such edge cases.

Some packages use a macro like this to protect against such problems:

  #define make_string(x) x ? Rf_mkString(x) : ScalarString(NA_STRING)

But I think it would make sense if this was the default behavior in
Rf_mkString and Rf_mkChar.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [R] R-3.3.0 Base: Windows compilation fails while building 'base' package

[Jeroen Ooms]

On Tue, May 17, 2016 at 5:32 AM, Venkat Ramakrishnan
 wrote:
> I am running the make from my D:\ partition, and I have created a \TMP
> directory
> under both C:\ and D:\.  I have also modified my environment variables TEMP
> and TMP both point to C:\TMP.

I think you actually need to set the 'TMPDIR' variable.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Suggestion: mkString(NULL) should be NA

[Jeroen Ooms]

On Thu, May 12, 2016 at 1:20 PM, Jeroen Ooms  wrote:
> I would like to propose that Rf_mkString(NULL) and Rf_mkChar(NULL)
> return NA rather than segfault.

An example implementation: https://git.io/vroxm

With this patch, mkChar(NULL), mkCharCE(NULL, ce) would return
NA_STRING rather than segfault at strlen(NULL). This automatically
fixes mkString(NULL) as well which wraps mkChar (See Rinlinedfuns.h).

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Suggestion: mkString(NULL) should be NA

[Jeroen Ooms]

On Tue, May 24, 2016 at 5:59 PM, Gabriel Becker  wrote:
> Shouldn't Rf_mkString(NULL) return (the c-level equivalent of) character()
> rather than the NA_character_?

No. It should still be safe to assume that mkString() always returns a
character vector of exactly length one. Anything else could lead to
type errors.

> An empty string and NULL aren't the same.

Exactly! So if you pass in an empty C string, you get an empty R
string, and if you pass in a null pointer you get NA.

Rf_mkString(NULL)   <--> NA
Rf_mkString("")   <--> ""

There is no ambiguity, and much better than segfaulting.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Suggestion: mkString(NULL) should be NA

[Jeroen Ooms]

On Wed, May 25, 2016 at 12:31 PM, Martin Maechler
 wrote:
> Better than segfaulting, yes, but really agree with Bill (and
> Gabe), also for Rf_mkChar(NULL):
> I think both functions should give an error in such a case
> rather than returning NA_character_
>
> It is an accident of some kind if they got NULL, no?

Not necessarily. A char* of NULL can be a string which is not
initiated or simply unavailable due to configuration.

The example from my original email was in curl package which exposes
the version string of libz that was used to build libcurl:

  mkString(data->libz_version)

This worked on all platforms that I tested. However a user found that
if libcurl was configured --without-libz (which is uncommon) the
libz_version string does not get set by libcurl and is always NULL. I
had not foreseen this and it would lead to a segfault.

I think making mkString() return NA for null strings lead to the most
robust behavior. Raising an exception seems a little harsh to me, as
there is no way the user would be able to recover from this, and there
might not be an actual problem at all.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Detecting user interrupts in R_tryEval

[Jeroen Ooms]

Is there any way to distinguish between an error and a user
interruption in R_tryEval? In both cases the ErrorOccurred argument is
set to 1. For my application I need a different action in case of a
SIGINT.

>From the source code I infer that R_tryEval basically wraps eval in
R_ToplevelExec, which returns TRUE if fun returns normally, FALSE if
it results in a jump to top level. However both an error and SIGINT
result in a jump. Is there an alternative method, or some method of
finding out which is the two happened after calling R_tryEval?

One thing I tried is see if R_curErrorBuf() is empty. However this is
unreliable because in the case of an interrupt, the error buffer
sometimes contains some old error message.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Nested tracing with custom callback

[Jeroen Ooms]

On Wed, Jul 13, 2016 at 5:20 AM, Evan James Patterson
 wrote:
>
> I would like to install a trace function that gets executed whenever *any* R 
> function is called. In Python, for example, this functionality is provided by 
> the `sys.settrace` function.

Maybe you can adapt from the covr package:
https://github.com/jimhester/covr/blob/master/vignettes/how_it_works.Rmd

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] package installation fails when symlink of same name exists

[Jeroen Ooms]

On Tue, Jul 19, 2016 at 6:46 PM, Kevin Ushey  wrote:
> R fails to install a package from source over a pre-existing package
> when the path to that package is a symlink, rather than a directory.
> ...
> I don't think anyone's reported this being an issue before

I ran into this as well a while back:
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16725

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Finalizer execution order question

[Jeroen Ooms]

Given an externalptr object 'pool' which protects an R object 'prot':

  # SEXP prot = (a dynamically updated list with handles)
  SEXP pool =  R_MakeExternalPtr(p, R_NilValue, prot);
  R_RegisterCFinalizerEx(pool, fin_pool, TRUE);

WRE explains that 'prot' remains in existence as long as 'pool' is
around. Does this also mean 'prot' still exists when the finalizer of
'pool' gets executed?

Long story: I am running into an issue with the next generation of the
curl package which uses dynamic pools of (many) http request handles.
Both the pool and the handles are externalptr objects with finalizers
that clean up after themselves.

When the pool goes out of scope, its finalizer has to loop over
pending handles to release them. However, to my surprise, the
finalizers of the handles get executed *before* the finalizer of
'pool'. Therefore the handles have destroyed themselves before I can
properly take hem out of the pool. This is exactly what I was trying
to prevent by putting the handles in a 'prot' list.

I wrote a simple package [1] that illustrates this issue. To run the example:

  devtools::install_github("jeroenooms/test")
  library(test)
  pool <- make_pool()
  show_handles(pool)
  rm(pool)
  gc()

What this example is supposed to illustrate is that even though 'prot'
gets protected from GC while the externalptr is around, the finalizers
in 'prot' have already executed when the externalptr gets finalized.

What is even stranger (to me) is that the SEXPs in 'prot' still seem
in tact during the finalizer of externalptr (ASAN is not giving
use-after-free warnings either). It's just that their finalizers have
already executed. So that leaves the pool finalizer in the odd
position of seeing only the post-mortem SEXP contents of the handles.
This does provide a workaround in my case, but can we rely on this?

So my question: is this expected behavior? What would be an
alternative approach to ensure that handles stay protected until the
pool has been cleaned and finalized (rather than running all
finalizers in the order they were registered)?


[1] https://github.com/jeroenooms/test

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Cluster: Various GCC, how important is consistency?

[Jeroen Ooms]

On Tue, Oct 18, 2016 at 1:44 AM, Paul Johnson  wrote:
>
> Administrator suggested I try to build with the GCC that is provided
> with the nodes, which is gcc-4.4.7.

Redhat provides an alternative compiler (gcc 5.3 based) in one of it's
opt-in repositories called "redhat developer toolkit" (RDT). In CentOS
you install it as follows:

  yum install -y centos-release-scl
  yum install -y devtoolset-4-gcc-c++

This compiler is specifically designed to be used alongside the EL6
stock gcc 4.4.7. It includes a simple 'enable' script which will put
RDT gcc and g++ in front of your PATH and LD_LIBRARY_PATH and so on.

So what I do on CentOS is install R from EPEL (built with stock gcc
4.4.7) and whenever I need to install an R package that uses e.g.
CXX11, simply start an R shell using the RDT compilers:

   source /opt/rh/devtoolset-4/enable
   R

>From what I have been able to test, this works pretty well (though I
am not a regular EL user). But I was able to build R packages that use
C++11 (such as feather) and once installed, these packages can be used
even in a regular R session (without RDT enabled).

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Support for signing R packages with GPG

[Jeroen Ooms]

I would like to propose adding experimental support for including a
PGP signature in R source packages. This would make it possible to
verify the identity of the package author and integrity of the package
sources.

There are two ways to implement this. Assuming GnuPG is on the PATH,
the CMD build script could call:

  gpg --clearsign MD5 -o MD5.gpg

Alternatively the 'gpg' R package provides a more portable method via
the gpgme C library. This method works on Windows / macOS as well.

  writeLines(gpg::gpg_sign("MD5"), "MD5.gpg")

Attached is an example implementation of the latter (also available at
https://git.io/vPb9G) which has been tested with several versions of
GnuPG. It exposes an optional flag for CMD build, i.e:

  R CMD build somepkg --sign
  R CMD build somepkg --sign=jeroen.o...@stat.ucla.edu

The --sign flag creates a signature for the MD5 file [1] in the source
package and saves it as MD5.gpg (similar to a Debian 'Release.gpg'
file [2]). Obviously the package author or build server needs to have
a suitable private key in the local keyring.


## Signature verification

Once R supports signed packages, we can develop a system to take
advantage of such signatures. The verification itself can easily be
implemented via 'gpg --verify' or via gpg::gpg_verify() and could be
performed without changes in R itself. The difficult part in GPG comes
from defining which peers should be trusted.

But even without a 'web of trust' there are several ways one can
immediately take advantage of signatures. For example, when a
installing a package update or dev-version of a package, we can verify
that the signature of the update matches that of the currently
installed package. This would prevent the type of attacks where an
intermediate party pushes a fake malicious update for a popular R
package via e.g. a hacked CRAN mirror.

Eventually, CRAN could consider allowing signatures as a secure
alternative to confirmation emails, and signing packages on the build
servers with a CRAN GPG key, similar to Debian repositories. For now,
at least establishing a format for (optionally) signing packages would
be a great first step.


[1] Eventually we should add SHA256 and SHA256.sig in addition to MD5
[2] 
https://cran.r-project.org/web/packages/gpg/vignettes/intro.html#debian_example
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Support for signing R packages with GPG

[Jeroen Ooms]

On Tue, Oct 25, 2016 at 7:22 PM, Martyn Plummer  wrote:
> Thanks Jeroen. The R Foundation has recently formed a working group to
> look into package authentication. There are basically two models. One
> is the GPG based model you describe; the other is to use X.509 as
> implemented in the PKI package. It's not yet clear which way to go but
> we are thinking about it.

I look forward to hearing what the working group comes up with. I
suppose if you go with x509, CRAN is going to perform CA duties?

Let me know if I can help with implementation, either via gpg or x509.
I am actively developing the openssl package which includes many more
x509 utilities, supporting all common key types (dsa, rsa, ec),
certificate bundles, ssl, etc. The main difference with PKI is that
openssl uses the native pem/der parsers from libssl which are more
robust and also recognize the less common formats, so that we don't
have to deal with parsing/decoding ASN.1 in R.

I will be happy to adapt/extend it further to fit the needs of the
workgroup and help this move forward.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Check package issue on mavericks

[Jeroen Ooms]

I just checked your package on a mavericks VM and it seems fine, so I
wouldn't worry about it. Probably some configuration issue on the CRAN
server, perhaps xquartz is outdated.

Unfortunately there is no easy way anymore to check on Mavericks.
Travis used to support it, but they have deprecated mavericks (along
with Apple and homebrew). Hopefully CRAN will update to the current
MacOS for the next release of R.




On Fri, Dec 9, 2016 at 3:56 PM, Gionata Bocci  wrote:
>
> Dear List,
>
>   I've realized that the "CRAN Checking Package" system returns some
> Warnings for my package TR8 under "r-release-osx-x86_64-mavericks"; all the
> warnings include the following message "Invalid MIT-MAGIC-COOKIE-1 key"
> which I find a little cryptic.
>   After some web searches, I understood that "MIT-MAGIC-COOKIE" is related
> to the X server and most of the messages reporting issues with that are
> related to R packages having problems with "mavericks" (either NOTES or
> Warnings resulting from the "checking process").
>   Not having a Mac available where I could try to explore this issue (and
> not being familiar with Mac in general), I've set up a Travis CI account in
> order to check whether the package was built and "checked --as-cran" on
> both Linux (my OS) and Mac OSX (Travis is using OS X versions >= 10.10): it
> looks like everything is fine, correctly built and checked.
>   I am thus writing to ask if other developers on the list have found and
> fixed this issue (so that I could fix that as well and tick the checkbox "I
> have fixed all problems shown on the package check page" before submitting
> a new version to CRAN).
>   Best,
>
>
> Gionata.
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Request: Increasing MAX_NUM_DLLS in Rdynload.c

[Jeroen Ooms]

On Tue, Dec 20, 2016 at 7:04 AM, Henrik Bengtsson
 wrote:
> On reason for hitting the MAX_NUM_DLLS (= 100) limit is because some
> packages don't unload their DLLs when they being unloaded themselves.

I am surprised by this. Why does R not do this automatically? What is
the case for keeping the DLL loaded after the package has been
unloaded? What happens if you reload another version of the same
package from a different library after unloading?

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [R] Problems when trying to install and load package "rzmq"

[Jeroen Ooms]

On Tue, Jan 3, 2017 at 3:53 PM, Whit Armstrong  wrote:
>
> I maintian the rzmq project.
>
> love to get it running on windows, but zmq doesn't play nicely with R's
> mingw.

It's fairly easy to link against the libraries from rwinlib:
https://github.com/rwinlib/zeromq. I'll send you a pull request later
this week to fix the binary packages for win+mac.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [R] Problems when trying to install and load package "rzmq"

[Jeroen Ooms]

For those following this thread: rzmq binary packages for Mac and
Windows are now available from CRAN.


On Tue, Jan 3, 2017 at 5:32 PM, Jeroen Ooms  wrote:
> On Tue, Jan 3, 2017 at 3:53 PM, Whit Armstrong  
> wrote:
>>
>> I maintian the rzmq project.
>>
>> love to get it running on windows, but zmq doesn't play nicely with R's
>> mingw.
>
> It's fairly easy to link against the libraries from rwinlib:
> https://github.com/rwinlib/zeromq. I'll send you a pull request later
> this week to fix the binary packages for win+mac.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] How to handle INT8 data

[Jeroen Ooms]

On Fri, Jan 20, 2017 at 6:09 PM, Murray Stokely  wrote:
> The lack of 64 bit integer support causes lots of problems when dealing
> with certain types of data where the loss of precision from coercing to 53
> bits with double is unacceptable.
>
> Two packages were developed to deal with this:  int64 and bit64.

Don't forget packages for large arbitrary large numbers such as Rmpfr
and openssl.

  x <- openssl::bignum("12345678987654321")
  x^10

The risk of storing int64 as a double (e.g. in bit64) is that it might
easily be mistaken for a completely different value via unclass() or
Rf_isNumeric() or so.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] parallel::mc*: Is it possible for a child process to know it is a fork?

[Jeroen Ooms]

On Tue, Jan 24, 2017 at 7:06 PM, Henrik Bengtsson
 wrote:
> When using multicore-forking of the parallel package, is it possible
> for a child process to know that it is a fork?

R internally uses R_isForkedChild to prevent certain operations within
the fork. However I don't think this is exported anywhere. You could
do something like:

  extern Rboolean R_isForkedChild;
  SEXP is_forked(){
return ScalarLogical(R_isForkedChild);
  }

But that won't be allowed on CRAN:

* checking compiled code ... NOTE
  Found non-API call to R: ‘R_isForkedChild’
  Compiled code should not call non-API entry points in R.

Another method would be to look at getppid(2) and getpgid(2) to lookup
the parent-id and group-id of the current process and test if it
matches that of the (parent) R process.

If you are only interested in limiting further parallelization within
the fork, perhaps you can simply use parallel::mcaffinity to restrict
the forked process to a single core.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] parallel and tempdir()

[Jeroen Ooms]

A forked process in mcparallel will wipe the (shared) tempdir when it
quits. I think this is almost always undesirable:

  file.exists(tempdir())
  parallel::mcparallel(q("no"))
  file.exists(tempdir()) # false
  file.create(tempfile()) # error

A simple solution might be to skip over R_CleanTempDir when
R_isForkedChild. Example patch: https://git.io/vDwyf

In addition, I would like to suggest to add an option to mcparallel to
set a custom tempdir in the fork to prevent potential conflicts. I
tried implementing this myself but found that setting R_TempDir is not
allowed by CMD check, and Sys_TempDir (which is what actually gets
wiped) is invisible behind attribute_hidden. So I suppose this may
only be done in base.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Registration of native routines

[Jeroen Ooms]

On Tue, Feb 14, 2017 at 5:25 PM, Prof Brian Ripley
 wrote:
>
> Registration has similar benefits to name spaces in R code:
>
> - it ensures that the routines used by .C, .Call etc are those in your 
> package (without needing a PACKAGE argument).
> - it avoids polluting the search space for native routines with those from 
> your package.
> - it checks the number of arguments passed to .Call/.External, and the number 
> and optionally the type for .C/.Fortran.
> - it finds native routines faster, especially if 10s of name spaces are 
> loaded.

Do these benefits also hold for packages that currently use useDynLib
exclusively in combination symbol names? E.g for the example from WRE:

   useDynLib(foo, myRoutine, myOtherRoutine)

Which is invoked via:

  .Call(myRoutine, x, y)

What ambiguity or pollution is introduced by foo:::myRoutine? Should
manually registering 'myRoutine' in C still be mandatory in this case?

It was nice having the NAMESPACE as the central declaration of
callable C routines. The "R_registerRoutines" system will require
maintaining additional C code which re-declares all callable C
functions from other compilation units. This introduces additional
complexity for package authors and might become a source of bugs when
we forget to update the registrations when C functions have changed.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Seeking advice regarding compilation of large libraries using RTools (Windows)

[Jeroen Ooms]

On Mon, Mar 6, 2017 at 6:33 AM, Richard Beare  wrote:
> I am working on the SimpleITK package for R. This is an enormous package
> that is largely automatically generated via a set of swig/json/lua magic,
> and is working well under linux and osx.

Is it available somewhere so we can try it?


> However we're having a lot of trouble with the Windows side. In fact, we are 
> struggling to get the base libraries to build using the RTools 3.4 toolchain, 
> even before the worrying about the R-specific parts.

What build environment do you use? The version of gcc with Rtools
should be ok, but the Rtools build utilities in the "bin" folder (in
particular 'make') are old and a frequent source of problems. However
for building external libs you can use other tools, for example those
from msys2. Just make sure you use gcc/g++ from Rtools.


> The current issue is very long time (possibly infinite) linking of dlls, or
> test executables. I've tried using a FAT32 file system for the build, as
> suggested by some old bug reports, but still have the issue.

On Windows you can avoid the run-time dll mess by building static libs
of external libraries. See rwinlib for examples:
https://github.com/rwinlib


> Any suggestions on where to turn next? Are cross compilers the next step?

Try building with msys2, but make sure to use gcc/g++ from Rtools by
setting the `CC` and `CXX` variables in the configure script. Cross
compiling will make things even more complicated because binaries
might not be compatible if your cross compiler has a different version
of gcc or has been configured for another exception model (seh/drawf).

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Experimental CXX_STD problem in R 3.4

[Jeroen Ooms]

R 3.4 has 'experimental' support for setting CXX_STD to CXX98 / CXX11
/ CXX14 / CXX17.

However on most platforms, the R configuration seems to leave the
CXX1Y and CXX1Z fields blank in "${R_HOME}/etc/Makeconf" (rather than
falling back on default CXX). Therefore specifying e.g CXX_STD= CXX14
will fail build with cryptic errors (due to compiling with CXX="")

I don't think this is intended? Some examples from r-devel on Windows:

CXX11: https://win-builder.r-project.org/R8gg703OQSq5/
CXX98: https://win-builder.r-project.org/mpVfXxk79FaN/
CXX14: https://win-builder.r-project.org/L3BSMgAk4cQ7/
CXX17: https://win-builder.r-project.org/3ETZXrgkg77I/

Similar problems appear on Linux. I think the problem is that Makeconf
contains e.g:

CXX1Z =
CXX1ZFLAGS =
CXX1ZPICFLAGS =
CXX1ZSTD =

When CXX_STD contains any other unsupported value (e.g. CXX24) R
simply falls back on the default CXX configuration. The same should
probably happen for e.g. CXX17 when CXX1Z is unset in Makeconf?

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Experimental CXX_STD problem in R 3.4

[Jeroen Ooms]

On Sun, Mar 19, 2017 at 9:09 PM, Martyn Plummer  wrote:
> I have just added some code to ensure that the compilation fails with an 
> informative error message if a specific C++ standard is requested but the 
> corresponding compiler has not been defined. Please test this.

Are you sure we shouldn't just fall back on a previous standard
instead of failing? For example if the package author has specified a
preference for CXX14 but the compiler only has CXX11, the package
might still build with -std=c++11 (given that C++14 is only a small
extension on the C++11 standard).

The current behavior (in R 3.3) for packages with "CXX_STD=CXX11" is
to fall back on CXX when the compiler does not have CXX1X. Will R-3.4
start failing these packages? This would affect many users on CentOS 6
(gcc 4.4.7).

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Experimental CXX_STD problem in R 3.4

[Jeroen Ooms]

On Sun, Mar 19, 2017 at 9:09 PM, Martyn Plummer  wrote:
> I have just added some code to ensure that the compilation fails with an 
> informative error message if a specific C++ standard is requested but the 
> corresponding compiler has not been defined. Please test this.

I can confirm that (at least on Windows) we get a proper error message
for CXX14 and CXX17, eg:

  * installing *source* package 'testcxx' ...
  ** libs
  Error in .shlib_internal(args) :
   C++14 standard requested but CXX1Y is not defined
  * removing 'C:/Program Files/R/R-devel/library/testcxx'

CXX98 still does not work though. It seems SHLIB_CXX98LD is undefined
on Windows?

  * installing *source* package 'testcxx' ...
  ** libs
  c:/Rtools/mingw_64/bin/g++  -std=gnu++98
-I"C:/PROGRA~1/R/R-devel/include" -DNDEBUG
-I"d:/Compiler/gcc-4.9.3/local330/include" -O2 -Wall  -mtune=core2
-c test.cpp -o test.o
  -shared -s -static-libgcc -o testcxx.dll tmp.def test.o
-Ld:/Compiler/gcc-4.9.3/local330/lib/x64
-Ld:/Compiler/gcc-4.9.3/local330/lib -LC:/PROGRA~1/R/R-devel/bin/x64
-lR
  -shared: not found
  no DLL was created
  ERROR: compilation failed for package 'testcxx'

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Hyperbolic tangent different results on Windows and Mac

[Jeroen Ooms]

This looks like a bug in mingw-w64 CRT. The problem can be produced
with C++ without R:

  #include 
  #include 
  #include 

  int main(){
std::cout << std::fixed;
std::complex z(356, 0);
std::cout << "tanh" << z << " = " << std::tanh(z)
 << " (tanh(356) = " << std::tanh(356) << ")\n";
  }

On OS-X we get:

  tanh(356.00,0.00) = (1.00,-0.00) (tanh(356) = 1.00)

But on Windows we get:

  tanh(356.00,0.00) = (nan,0.00) (tanh(356) = 1.00)

I was also able to reproduce the problem with gcc 6.3 in msys2 so it
has not been fixed upstream. You should file a bug report for
mingw-w64.

FWIF, we have run into NaN edge-case bugs before with mingw-w64.

 - 
https://sourceforge.net/p/mingw-w64/mingw-w64/ci/6617ebd5fc6b790c80071d5b1d950e737fc670e1/
 - 
https://github.com/wch/r-source/commit/e9aaf8fdeddf27c2a9078cd214a41475c8ff6f40

I am cc'ing Ray Donnelly who is an expert on mingw-w64.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Hyperbolic tangent different results on Windows and Mac

[Jeroen Ooms]

For future reference:
https://sourceforge.net/p/mingw-w64/mailman/message/35747206/

On Wed, Mar 22, 2017 at 2:12 PM, Jeroen Ooms  wrote:
> This looks like a bug in mingw-w64 CRT. The problem can be produced
> with C++ without R:
>
>   #include 
>   #include 
>   #include 
>
>   int main(){
> std::cout << std::fixed;
> std::complex z(356, 0);
> std::cout << "tanh" << z << " = " << std::tanh(z)
>  << " (tanh(356) = " << std::tanh(356) << ")\n";
>   }
>
> On OS-X we get:
>
>   tanh(356.00,0.00) = (1.00,-0.00) (tanh(356) = 1.00)
>
> But on Windows we get:
>
>   tanh(356.00,0.00) = (nan,0.00) (tanh(356) = 1.00)
>
> I was also able to reproduce the problem with gcc 6.3 in msys2 so it
> has not been fixed upstream. You should file a bug report for
> mingw-w64.
>
> FWIF, we have run into NaN edge-case bugs before with mingw-w64.
>
>  - 
> https://sourceforge.net/p/mingw-w64/mingw-w64/ci/6617ebd5fc6b790c80071d5b1d950e737fc670e1/
>  - 
> https://github.com/wch/r-source/commit/e9aaf8fdeddf27c2a9078cd214a41475c8ff6f40
>
> I am cc'ing Ray Donnelly who is an expert on mingw-w64.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] potential bug in attribute handling for externalptr

[Jeroen Ooms]

On Fri, Apr 14, 2017 at 11:27 PM, Patrick Perry  wrote:
> Is the following expected behavior? I would expect that modifying y's 
> attributes should not affect x.

Yes this is expected. Environments and external-ptr objects are never
copied on assignment, hence in your example 'x' and 'y' are the same
object. See e.g. ?reg.finalizer or section 5.13 of "writing r
extensions" on external pointers and weak references.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] tempdir() may be deleted during long-running R session

[Jeroen Ooms]

On Tue, Apr 25, 2017 at 1:00 PM, Martin Maechler
 wrote:
> As I've found it is not at all hard to add an option which
> checks the existence and if the directory is no longer "valid",
> tries to recreate it (and if it fails doing that it calls the
> famous R_Suicide(), as it does when R starts up and tempdir()
> cannot be initialized correctly).

Perhaps this can also fix the problem with mcparallel deleting the
tempdir() when one of its children dies:

  file.exists(tempdir()) #TRUE
  parallel::mcparallel(q('no'))
  file.exists(tempdir()) # FALSE

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Error messages in replayPlot()

[Jeroen Ooms]

I was wondering if there is something that can be done to improve
error messages when replaying a recorded plot. For example a graphics
device that is too small usually results in a helpful error message:

 png(height = 100)
 plot(1)
 # Error in plot.new() : figure margins too large
 dev.off()

However when this happens when replaying a recorded plot, the error
message is not so helpful.

 myplot <- evaluate::evaluate("plot(1)")[[2]]
 png(height = 100)
 replayPlot(myplot)
 # Error in replayPlot(x) : invalid graphics state
 dev.off()

A more informative error message that hints at what exactly is invalid
about the graphics state would be very helpful in this case.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Fortran programs not writing stdout on windows

[Jeroen Ooms]

A user has reported an issue that appears when a fortran executable is
called via R on Windows. I am unsure if this expected behavior or a
bug in Fortran or in how R calls Windows executables.

The problem is that when the fortran program is called from R, stdout
gets written to a file "fort.6" instead of stdout. When the same
executable is called from the terminal it works fine and prints to
stdout. This unexpected behavior is unfortunate for R wrappers that
rely on captured output.

A minimal example is available from github [1] and can be installed with

   devtools::install_github("jeroen/ftest")
   ftest::hello()

When running ftest::hello() on linux, R will properly capture output.
However on Windows it will return an empty string, and a file 'fort.6'
gets created in the working directory instead.

The executables can be found in: system.file("bin", package = "ftest")

Interestingly if we open a command line terminal and run the same
executable it prints output to stdout. So perhaps it has to do with
the way R invokes CreateProcess() on Windows?

[1] https://github.com/jeroen/ftest

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Separate packages per windows subarch in repository

[Jeroen Ooms]

On Thu, Jul 20, 2017 at 1:55 PM, Iago Mosqueira
 wrote:
>
> I maintain a repository of R packages, where some of them contain
> executable binaries. I need to separate those compiled for 32 and 64 bit in
> Windows.

Have a look at the antiword package. It has a simple makevars which
builds antiword$(WIN) executable which is just the 'antiword' on unix,
and antiword32.exe + antiword64.exe on multiarch windows.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] readLines() segfaults on large file & question on how to work around

[Jeroen Ooms]

On Sat, Sep 2, 2017 at 8:58 PM, Jennifer Lyon  wrote:
> I have a 2.1GB JSON file. Typically I use readLines() and
> jsonlite:fromJSON() to extract data from a JSON file.

If your data consists of one json object per line, this is called
'ndjson'. There are several packages specialized to read ndjon files:

 - corpus::read_ndjson
 - ndjson::stream_in
 - jsonlite::stream_in

In particular the 'corpus' package handles large files really well
because it has an option to memory-map the file instead of reading all
of its data into memory.

If the data is too large to read, you can preprocess it using
https://stedolan.github.io/jq/ to extract the fields that you need.

You really don't need hadoop/spark/etc for this.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Configuration Variable: Maximum Memory Allocation per R Instance

[Jeroen Ooms]

On Sun, Sep 17, 2017 at 12:39 AM, Juan Telleria  wrote:
> Dear R Developers,
>
> In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the
> innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> amount of RAM which can be used by a Server Instance.

Memory is not controlled by R itself because packages may malloc()
directly. However most operating systems have features to limit
resources of a given process. The CRAN package 'unix' has wrappers for
posix setrlimit [1] e.g. unix::rlimit_as() limits address space. This
works pretty well, however I found that the way memory is managed and
counted varies a lot per OS and malloc implementation.

You can also set rlimits on a single evaluation via the rlimit
parameter in sys::eval_safe().


[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/getrlimit.html

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel