[Rd] Intermittent crashes with inset `[<-` command

[Brian Montgomery via R-devel]

The following code crashes after about 300 iterations on my x86_64-w64-mingw32 
machine on R 3.5.2 --vanilla.  
Others have duplicated this (see 
https://github.com/tidyverse/magrittr/issues/190 if necessary), but I don't 
know how machine/OS-dependent it may be.  
If it doesn't crash for you, please try increasing the length of the x vector.

Substituting the commented-out line for the one below it works correctly 
(prints out 1:1000 and ends normally) every time.

x <- 1:20
y <- rep(letters[1:5], length(x) / 5L)
for (i in 1:1000) {
  # x[y == 'a'] <- x[y == 'b']
  x <- `[<-`(x, y == 'a', x[y == 'b'])
  cat(i, '')
}
cat('\n')

The point of using this syntax is to make it work better with pipes, but the 
errors occur without pipes or magrittr.

Thank you for your help!

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Compile R to WebAssembly / Emscripten?

[Scott Chamberlain]

Recently Michael Droettboom at Mozilla was asking around about
doing for R what they've done for Python here
https://github.com/iodide-project/pyodide
with the Iodide project

S

On Tue, Feb 26, 2019 at 11:47 AM Gabriel Becker 
wrote:

> As I recall, the major blocker is that R links against a number of other
> things (notably BLAS, pcre, etc) so while technically possible (?) I
> suppose, the universe of things you'd have to compile over and then get
> working is much larger than just the R internals.
>
> I think most people who consider this (including me years ago, as well as
> the poster of Gabor's message to rdevel) hit that point and then go try to
> find a less herculean task to pursue.
>
> ~G
>
> On Wed, Feb 20, 2019 at 12:57 AM Gábor Csárdi 
> wrote:
>
> > This was some time ago:
> > https://stat.ethz.ch/pipermail/r-devel/2013-May/066724.html
> >
> > So probably not hopeless, but I would think it is a lot of work.
> >
> > Gabor
> >
> > On Wed, Feb 20, 2019 at 8:17 AM Todd Wilder  wrote:
> > >
> > > Has anyone attempted to compile R (probably without any OS bindings) to
> > > WebAssembly / Emscripten? If so, how far did you get? (would be crazy
> > > awesome if you could get all the way to a ggplot bitmap output). If
> not,
> > is
> > > this a waste of time or is there some daylight to doing this?
> > >
> > > [[alternative HTML version deleted]]
> > >
> > > __
> > > R-devel@r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[Gabriel Becker]

Hi Brian,

I don't have a windows machine, but on a modern macbook pro I'm not able to
get this to crash in 3.5.1 or a very recent built-from-source R-devel.  I
increased the length of x by 2 orders of magnitude but that just made the
loop take a lot longer to successfully run in both R versions i tested.

So seems like there is some OS/machine dependency at work here (?).

Best,
~G

On Wed, Feb 27, 2019 at 12:01 AM Brian Montgomery via R-devel <
r-devel@r-project.org> wrote:

> The following code crashes after about 300 iterations on
> my x86_64-w64-mingw32 machine on R 3.5.2 --vanilla.
> Others have duplicated this (see
> https://github.com/tidyverse/magrittr/issues/190 if necessary), but I
> don't know how machine/OS-dependent it may be.
> If it doesn't crash for you, please try increasing the length of the x
> vector.
>
> Substituting the commented-out line for the one below it works correctly
> (prints out 1:1000 and ends normally) every time.
>
> x <- 1:20
> y <- rep(letters[1:5], length(x) / 5L)
> for (i in 1:1000) {
>   # x[y == 'a'] <- x[y == 'b']
>   x <- `[<-`(x, y == 'a', x[y == 'b'])
>   cat(i, '')
> }
> cat('\n')
>
> The point of using this syntax is to make it work better with pipes, but
> the errors occur without pipes or magrittr.
>
> Thank you for your help!
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[Serguei Sokol]

On 26/02/2019 05:18, Brian Montgomery via R-devel wrote:

The following code crashes after about 300 iterations on my x86_64-w64-mingw32 
machine on R 3.5.2 --vanilla.
Others have duplicated this (see 
https://github.com/tidyverse/magrittr/issues/190 if necessary), but I don't 
know how machine/OS-dependent it may be.

It crashes too on my Mageia6 (RPM based Linux distribution):
 184 185 186 187
 *** caught segfault ***
address 0x70002, cause 'memory not mapped'

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

The crash can happen at different moments, sometimes after i=187 like in 
the example above, sometimes after i=915. The error is not always 
segfault. It can also be


915 Error in `[<-`(x, y == "a", x[y == "b"]) : replacement has length zero

or

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Error in 
`[<-`(x, y == "a", x[y == "b"]) :
  types (de raw a integer) incompatibles dans l'ajustement 
d'affectation de type


(sorry, this crash was in french locale)

Hoping this helps.
Serguei.

>  sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Mageia 6

Matrix products: default
BLAS/LAPACK: /home/opt/OpenBLAS/lib/libopenblas_sandybridge-r0.3.3.so

locale:
[1] C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods base

loaded via a namespace (and not attached):
[1] compiler_3.5.2


If it doesn't crash for you, please try increasing the length of the x vector.

Substituting the commented-out line for the one below it works correctly 
(prints out 1:1000 and ends normally) every time.

x <- 1:20
y <- rep(letters[1:5], length(x) / 5L)
for (i in 1:1000) {
   # x[y == 'a'] <- x[y == 'b']
   x <- `[<-`(x, y == 'a', x[y == 'b'])
   cat(i, '')
}
cat('\n')

The point of using this syntax is to make it work better with pipes, but the 
errors occur without pipes or magrittr.

Thank you for your help!

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Improved Data Aggregation and Summary Statistics in R

[Iñaki Ucar]

On Wed, 27 Feb 2019 at 09:02, Sebastian Martin Krantz
 wrote:
>
> Dear Developers,
>
> Having spent time developing and thinking about how data aggregation and
> summary statistics can be enhanced in R, I would like to present my
> ideas/efforts in the form of two commands:
>
> The first, which for now I called 'collap', is an upgrade of aggregate that
> accommodates and extends the functionality of aggregate in various
> respects, most importantly to work with multilevel and multi-type data,
> multiple function calls, highly customized aggregation tasks, a much
> greater flexibility in the passing of inputs and tidy output.
>
> The second function, 'qsu', is an advanced and flexible summary command for
> cross-sectional and multilevel (panel) data (i.e. it can provide overall,
> between and within entities statistics, and allows for grouping, custom
> functions and transformations). It also provides a quick method to compute
> and output within-transformed data.
>
> Both commands are efficiently built from core R, but provide for optional
> integration with data.table, which renders them extremely fast on large
> datasets. An explanation of the syntax, a demonstration and benchmark
> results are provided in the attached vignette.

Looks interesting. Sorry if it's there and I didn't find it: is there
any package implementing these functions so that we can try them?

Iñaki

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[Iñaki Ucar]

On Wed, 27 Feb 2019 at 09:51, Serguei Sokol  wrote:
>
> On 26/02/2019 05:18, Brian Montgomery via R-devel wrote:
> > The following code crashes after about 300 iterations on my 
> > x86_64-w64-mingw32 machine on R 3.5.2 --vanilla.
> > Others have duplicated this (see 
> > https://github.com/tidyverse/magrittr/issues/190 if necessary), but I don't 
> > know how machine/OS-dependent it may be.
> It crashes too on my Mageia6 (RPM based Linux distribution):
>   184 185 186 187
>   *** caught segfault ***
> address 0x70002, cause 'memory not mapped'

I can reproduce it too. This is the output from valgrind (sessionInfo below):

==3296== Invalid read of size 1
==3296==at 0x4A2D7F7: UnknownInlinedFun (Rinlinedfuns.h:542)
==3296==by 0x4A2D7F7: VectorAssign (subassign.c:658)
==3296==by 0x4A30540: do_subassign_dflt (subassign.c:1641)
==3296==by 0x4A338F2: do_subassign (subassign.c:1571)
==3296==by 0x49769A1: bcEval (eval.c:6795)
==3296==by 0x498B415: R_compileAndExecute (eval.c:1407)
==3296==by 0x498B985: do_for (eval.c:2185)
==3296==by 0x49848A8: Rf_eval (eval.c:691)
==3296==by 0x49B5131: Rf_ReplIteration (main.c:258)
==3296==by 0x49B5131: Rf_ReplIteration (main.c:198)
==3296==by 0x49B54F0: R_ReplConsole (main.c:308)
==3296==by 0x49B55AF: run_Rmainloop (main.c:1082)
==3296==by 0x1090AE: main (Rmain.c:29)
==3296==  Address 0x1dafab90 is 0 bytes inside a block of size 160,048 free'd
==3296==at 0x4839A0C: free (vg_replace_malloc.c:540)
==3296==by 0x49BCA56: ReleaseLargeFreeVectors (memory.c:1055)
==3296==by 0x49BCA56: RunGenCollect (memory.c:1825)
==3296==by 0x49BCA56: R_gc_internal (memory.c:2998)
==3296==by 0x49BCA56: R_gc_internal (memory.c:2964)
==3296==by 0x49BFB2C: Rf_allocVector3 (memory.c:2682)
==3296==by 0x49C09FC: UnknownInlinedFun (Rinlinedfuns.h:577)
==3296==by 0x49C09FC: R_alloc (memory.c:2197)
==3296==by 0x4A377F5: logicalSubscript (subscript.c:575)
==3296==by 0x4A377F5: logicalSubscript (subscript.c:503)
==3296==by 0x4A3A8D3: Rf_makeSubscript (subscript.c:994)
==3296==by 0x4A2D63D: VectorAssign (subassign.c:656)
==3296==by 0x4A30540: do_subassign_dflt (subassign.c:1641)
==3296==by 0x4A338F2: do_subassign (subassign.c:1571)
==3296==by 0x49769A1: bcEval (eval.c:6795)
==3296==by 0x498B415: R_compileAndExecute (eval.c:1407)
==3296==by 0x498B985: do_for (eval.c:2185)
==3296==  Block was alloc'd at
==3296==at 0x483880B: malloc (vg_replace_malloc.c:309)
==3296==by 0x49C0031: Rf_allocVector3 (memory.c:2713)
==3296==by 0x4A3B041: UnknownInlinedFun (Rinlinedfuns.h:577)
==3296==by 0x4A3B041: Rf_ExtractSubset (subset.c:115)
==3296==by 0x4A3DA8A: VectorSubset (subset.c:198)
==3296==by 0x4A3DA8A: do_subset_dflt (subset.c:823)
==3296==by 0x4A3FCAA: do_subset (subset.c:661)
==3296==by 0x49848A8: Rf_eval (eval.c:691)
==3296==by 0x4989600: Rf_evalListKeepMissing (eval.c:2955)
==3296==by 0x4A3390B: R_DispatchOrEvalSP (subassign.c:1535)
==3296==by 0x4A3390B: do_subassign (subassign.c:1567)
==3296==by 0x49769A1: bcEval (eval.c:6795)
==3296==by 0x498B415: R_compileAndExecute (eval.c:1407)
==3296==by 0x498B985: do_for (eval.c:2185)
==3296==by 0x49848A8: Rf_eval (eval.c:691)
==3296==
==3296== Invalid read of size 1
==3296==at 0x4A2E2C0: UnknownInlinedFun (Rinlinedfuns.h:189)
==3296==by 0x4A2E2C0: UnknownInlinedFun (Rinlinedfuns.h:554)
==3296==by 0x4A2E2C0: VectorAssign (subassign.c:658)
==3296==by 0x4A30540: do_subassign_dflt (subassign.c:1641)
==3296==by 0x4A338F2: do_subassign (subassign.c:1571)
==3296==by 0x49769A1: bcEval (eval.c:6795)
==3296==by 0x498B415: R_compileAndExecute (eval.c:1407)
==3296==by 0x498B985: do_for (eval.c:2185)
==3296==by 0x49848A8: Rf_eval (eval.c:691)
==3296==by 0x49B5131: Rf_ReplIteration (main.c:258)
==3296==by 0x49B5131: Rf_ReplIteration (main.c:198)
==3296==by 0x49B54F0: R_ReplConsole (main.c:308)
==3296==by 0x49B55AF: run_Rmainloop (main.c:1082)
==3296==by 0x1090AE: main (Rmain.c:29)
==3296==  Address 0x1dafab90 is 0 bytes inside a block of size 160,048 free'd
==3296==at 0x4839A0C: free (vg_replace_malloc.c:540)
==3296==by 0x49BCA56: ReleaseLargeFreeVectors (memory.c:1055)
==3296==by 0x49BCA56: RunGenCollect (memory.c:1825)
==3296==by 0x49BCA56: R_gc_internal (memory.c:2998)
==3296==by 0x49BCA56: R_gc_internal (memory.c:2964)
==3296==by 0x49BFB2C: Rf_allocVector3 (memory.c:2682)
==3296==by 0x49C09FC: UnknownInlinedFun (Rinlinedfuns.h:577)
==3296==by 0x49C09FC: R_alloc (memory.c:2197)
==3296==by 0x4A377F5: logicalSubscript (subscript.c:575)
==3296==by 0x4A377F5: logicalSubscript (subscript.c:503)
==3296==by 0x4A3A8D3: Rf_makeSubscript (subscript.c:994)
==3296==by 0x4A2D63D: VectorAssign (subassign.c:656)
==3296==by 0x4A30540: do_subassign_dflt (subassign.c:1641)
==3296==by 0x4A338F2: do_subassign (subassi

Re: [Rd] Improved Data Aggregation and Summary Statistics in R

[Joris Meys]

Dear Sebastian,

Initially I was a bit hesitant to think about yet another way to summarize
data, but your illustrations convinced me this is actually a great addition
to the toolset currently available in different R packages. Many of us have
written custom functions to get the required tables for specific data sets,
but this would reduce that effort to simply using the right collap() call.

Like Inaki, I'm very interested in trying it out if you have the code
available somewhere.

Cheers
Joris





On Wed, Feb 27, 2019 at 9:01 AM Sebastian Martin Krantz <
sebastian.kra...@graduateinstitute.ch> wrote:

> Dear Developers,
>
> Having spent time developing and thinking about how data aggregation and
> summary statistics can be enhanced in R, I would like to present my
> ideas/efforts in the form of two commands:
>
> The first, which for now I called 'collap', is an upgrade of aggregate that
> accommodates and extends the functionality of aggregate in various
> respects, most importantly to work with multilevel and multi-type data,
> multiple function calls, highly customized aggregation tasks, a much
> greater flexibility in the passing of inputs and tidy output.
>
> The second function, 'qsu', is an advanced and flexible summary command for
> cross-sectional and multilevel (panel) data (i.e. it can provide overall,
> between and within entities statistics, and allows for grouping, custom
> functions and transformations). It also provides a quick method to compute
> and output within-transformed data.
>
> Both commands are efficiently built from core R, but provide for optional
> integration with data.table, which renders them extremely fast on large
> datasets. An explanation of the syntax, a demonstration and benchmark
> results are provided in the attached vignette.
>
> Since both commands accommodate existing functionality while adding
> significant basic functionality, I though that their addition to the stats
> package would be a worthwhile consideration. I am happy for your feedback.
>
> Best regards,
>
> Sebastian Krantz
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>


-- 
Joris Meys
Statistical consultant

Department of Data Analysis and Mathematical Modelling
Ghent University
Coupure Links 653, B-9000 Gent (Belgium)


---
Biowiskundedagen 2018-2019
http://www.biowiskundedagen.ugent.be/

---
Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[Rui Barradas]

Hello,

I can also reproduce this, R 3.5.2 on Ubuntu 18.04 LTS.


First run gives error after 148.

After 148:

Error in `[<-`(x, y == "a", x[y == "b"]) :
  substituto tem comprimento zero
Execução interrompida

Translation:

replacement has length zero
Execution stopped


Second run gives a different error, I changed the script to start with a 
sessionInfo() instruction, everything else is the same as in the OP.


After 180:

 *** caught segfault ***
address 0x70002, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...


Several runs with or without the sessionInfo() always give the two 
different errors above.


The sessionInfo() was the following.

sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.1 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=pt_PT.UTF-8   LC_NUMERIC=C
 [3] LC_TIME=pt_PT.UTF-8LC_COLLATE=pt_PT.UTF-8
 [5] LC_MONETARY=pt_PT.UTF-8LC_MESSAGES=pt_PT.UTF-8
 [7] LC_PAPER=pt_PT.UTF-8   LC_NAME=C
 [9] LC_ADDRESS=C   LC_TELEPHONE=C
[11] LC_MEASUREMENT=pt_PT.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.5.2


Hope this helps,

Rui Barradas


Às 08:46 de 27/02/2019, Serguei Sokol escreveu:

On 26/02/2019 05:18, Brian Montgomery via R-devel wrote:
The following code crashes after about 300 iterations on 
my x86_64-w64-mingw32 machine on R 3.5.2 --vanilla.
Others have duplicated this 
(see https://github.com/tidyverse/magrittr/issues/190 if necessary), 
but I don't know how machine/OS-dependent it may be.

It crashes too on my Mageia6 (RPM based Linux distribution):
  184 185 186 187
  *** caught segfault ***
address 0x70002, cause 'memory not mapped'

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

The crash can happen at different moments, sometimes after i=187 like in 
the example above, sometimes after i=915. The error is not always 
segfault. It can also be


915 Error in `[<-`(x, y == "a", x[y == "b"]) : replacement has length zero

or

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Error in 
`[<-`(x, y == "a", x[y == "b"]) :
   types (de raw a integer) incompatibles dans l'ajustement 
d'affectation de type


(sorry, this crash was in french locale)

Hoping this helps.
Serguei.

 >  sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Mageia 6

Matrix products: default
BLAS/LAPACK: /home/opt/OpenBLAS/lib/libopenblas_sandybridge-r0.3.3.so

locale:
[1] C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods base

loaded via a namespace (and not attached):
[1] compiler_3.5.2

If it doesn't crash for you, please try increasing the length of the x 
vector.


Substituting the commented-out line for the one below it works 
correctly (prints out 1:1000 and ends normally) every time.


x <- 1:20
y <- rep(letters[1:5], length(x) / 5L)
for (i in 1:1000) {
   # x[y == 'a'] <- x[y == 'b']
   x <- `[<-`(x, y == 'a', x[y == 'b'])
   cat(i, '')
}
cat('\n')

The point of using this syntax is to make it work better with pipes, 
but the errors occur without pipes or magrittr.


Thank you for your help!

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] stopifnot

[Martin Maechler]

> Suharto Anggono Suharto Anggono via R-devel 
> on Sun, 24 Feb 2019 14:22:48 + writes:

>> From https://github.com/HenrikBengtsson/Wishlist-for-R/issues/70 :
> ... and follow up note from 2018-03-15: Ouch... in R-devel, stopifnot() 
has become yet 4-5 times slower;

> ...
> which is due to a complete rewrite using tryCatch() and 
withCallingHandlers().


>> From https://stat.ethz.ch/pipermail/r-devel/2017-May/074256.html , it 
seems that 'tryCatch' was used to avoid the following example from giving error 
message with 'eval' call and 'withCallingHandlers' was meant to handle similar 
case for warning.
> tst <- function(y) { stopifnot(is.numeric(y)); y+ 1 }
> try(tst())

> However,
> withCallingHandlers(,
> warning = function(w) { w$call <- cl.i; w })
> actally has no effect. In current code of function 'stopifnot', 'eval' is 
used only in handling stopifnot(exprs=) . The warning message from
> stopifnot(exprs={warning()})
> has 'eval' call:
> In eval(cl.i, envir = envir) : 

> This may work.
> withCallingHandlers(,
> warning = function(w) {
> w$call <- cl.i; warning(w); invokeRestart("muffleWarning") })


> Current documentation says:
> Since R version 3.5.0, expressions are evaluated sequentially, and hence 
evaluation stops as soon as there is a "non-TRUE", asnindicated by the above 
conceptual equivalence statement. Further, when such an expression signals an 
error or warning, its conditionCall() no longer contains the full stopifnot 
call, but just the erroneous expression.

> I assume that "no longer contains ..." is supposed to be the effect of 
the use of 'withCallingHandlers' and 'tryCatch' in 'stopifnot'.

> Actually, "contains the full stopifnot call" is not always the case in R 
before version 3.5.0. Normally, the call is the "innermost context".

Thank you Suharto, for thinking about these issues and being
constructive, trying to improve the current state.

Unfortunately, I do not quite understand what you are trying to
say here.

The main somewhat recent changes to stopifnot() have been (in
inverse time ordering)

1) Do what the documentation always promised, namely eval() the
   expressions one by one, and stop evaluation as soon as one of
   them is not all(.) TRUE.
   For that reason, the previously used idiom   'list(...)'
   is a no go, as "of course", it evaluates all the expressions in '...'

2) Try to ensure that warning() and stop()/error messages are
   shown the same {as closely as feasible}  to how they are
   shown outside of stopifnot(.)
  ==> partly the topic of this e-mail.

3) [2.5 years ago:] stopifnot() became smart about all.equal(.) expressions,
  showing the all.equal() string if it was not TRUE:
  In older R versions (<= 3.3.x ?), we had

  > stopifnot(all.equal(pi, 3.1415))
 Error: all.equal(pi, 3.1415) is not TRUE

  where as in R (>= 3.4.0 at least):

  > stopifnot(all.equal(pi, 3.1415)) 
  Error: pi and 3.1415 are not equal:
Mean relative difference: 2.949255e-05


One example of what I meant with the above documentation ("no
longer contains")  is the following:

In R 3.5.x, 

   > lf <- list(fm = y ~ f(x), osf = ~ sin(x))
   > stopifnot(identical(deparse(lf), deparse(lf, control="all")))
   Warning message:
   In deparse(lf, control = "all") : deparse may be incomplete
   > 

If I change the calling handler to use the
invokeRestart("muffleWarning") which I understand you are
proposing, then the message becomes

   Warning message:
   In identical(deparse(lf, control = "all"), deparse(lf)) :
 deparse may be incomplete

which is less useful as I can no longer see directly which of
the deparse() produced the warning.

> Example:
> stopifnot((1:2) + (1:3) > 0)
> Warning message:
> In (1:2) + (1:3) :
>   longer object length is not a multiple of shorter object length

Which is the good answer
(whereas also showing "> 0" in the warning is slightly off).

Again, if I'd use the  ..muffleWarning.. code instead, the above
would change to the worse

 Warning message:
 In (1:2) + (1:3) > 0 :
   longer object length is not a multiple of shorter object length

which "wrongly includes the '> 0'.
So I guess I really don't understand what you are proposing, or
would like to change  ...


> Example that gives error:
> stopifnot(is.na(log("a")))
> R 3.5.0:
> R 3.3.2:

That's a good one: we want the error message *not to* mention
is.na(.) but just 'log': i.e.,

We'd like  [ R versions <= 3.4.4 ] :

> stopifnot(is.na(log("a")))
Error in log("a") : non-numeric argument to mathematical function

as opposed to [ R version >= 3.5.0 ] :

> stopifnot(is.na(log("a")))
Error in is.na(log("a")) : non-numeric argument to mathematical function

-

Again, I'm sure I partly failed to understand what you said in
your e-mail and apologize for that.

Of course, I'm ha

Re: [Rd] Improved Data Aggregation and Summary Statistics in R

[Sebastian Martin Krantz]

Dear Iñaki and Joris,

thank you for the positive feedback! I had attached a code file to the
post, but apparently it was removed.
I will attach it again to this e-mail, otherwise both vignette and code can
be downloaded from the following link:
https://www.dropbox.com/sh/s0k1tiz7el55g1q/AACpri-nruXjcMwUnNcHoycKa?dl=0
Best,
Sebastian

On Wed, 27 Feb 2019 at 11:14, Joris Meys  wrote:

> Dear Sebastian,
>
> Initially I was a bit hesitant to think about yet another way to summarize
> data, but your illustrations convinced me this is actually a great addition
> to the toolset currently available in different R packages. Many of us have
> written custom functions to get the required tables for specific data sets,
> but this would reduce that effort to simply using the right collap() call.
>
> Like Inaki, I'm very interested in trying it out if you have the code
> available somewhere.
>
> Cheers
> Joris
>
>
>
>
>
> On Wed, Feb 27, 2019 at 9:01 AM Sebastian Martin Krantz <
> sebastian.kra...@graduateinstitute.ch> wrote:
>
>> Dear Developers,
>>
>> Having spent time developing and thinking about how data aggregation and
>> summary statistics can be enhanced in R, I would like to present my
>> ideas/efforts in the form of two commands:
>>
>> The first, which for now I called 'collap', is an upgrade of aggregate
>> that
>> accommodates and extends the functionality of aggregate in various
>> respects, most importantly to work with multilevel and multi-type data,
>> multiple function calls, highly customized aggregation tasks, a much
>> greater flexibility in the passing of inputs and tidy output.
>>
>> The second function, 'qsu', is an advanced and flexible summary command
>> for
>> cross-sectional and multilevel (panel) data (i.e. it can provide overall,
>> between and within entities statistics, and allows for grouping, custom
>> functions and transformations). It also provides a quick method to compute
>> and output within-transformed data.
>>
>> Both commands are efficiently built from core R, but provide for optional
>> integration with data.table, which renders them extremely fast on large
>> datasets. An explanation of the syntax, a demonstration and benchmark
>> results are provided in the attached vignette.
>>
>> Since both commands accommodate existing functionality while adding
>> significant basic functionality, I though that their addition to the stats
>> package would be a worthwhile consideration. I am happy for your feedback.
>>
>> Best regards,
>>
>> Sebastian Krantz
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
>
> --
> Joris Meys
> Statistical consultant
>
> Department of Data Analysis and Mathematical Modelling
> Ghent University
> Coupure Links 653, B-9000 Gent (Belgium)
>
> 
>
> ---
> Biowiskundedagen 2018-2019
> http://www.biowiskundedagen.ugent.be/
>
> ---
> Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Improved Data Aggregation and Summary Statistics in R

[Duncan Murdoch]

On 26/02/2019 8:25 a.m., Sebastian Martin Krantz wrote:

Dear Developers,

Having spent time developing and thinking about how data aggregation and
summary statistics can be enhanced in R, I would like to present my
ideas/efforts in the form of two commands:

The first, which for now I called 'collap', is an upgrade of aggregate that
accommodates and extends the functionality of aggregate in various
respects, most importantly to work with multilevel and multi-type data,
multiple function calls, highly customized aggregation tasks, a much
greater flexibility in the passing of inputs and tidy output.

The second function, 'qsu', is an advanced and flexible summary command for
cross-sectional and multilevel (panel) data (i.e. it can provide overall,
between and within entities statistics, and allows for grouping, custom
functions and transformations). It also provides a quick method to compute
and output within-transformed data.

Both commands are efficiently built from core R, but provide for optional
integration with data.table, which renders them extremely fast on large
datasets. An explanation of the syntax, a demonstration and benchmark
results are provided in the attached vignette.

Since both commands accommodate existing functionality while adding
significant basic functionality, I though that their addition to the stats
package would be a worthwhile consideration. I am happy for your feedback.


Generally the R Core group is reluctant to incorporate new functions 
into the base packages.  Each function that is added adds to their work, 
and they already have too much to do.  (I am no longer a member of R 
Core, but I don't think things have changed since I retired.)


It is much easier for them if volunteers publish functions themselves, 
via contributed packages.


Nowadays Github provides a very convenient platform on which you can 
develop a package containing your functions.  If other users find bugs 
or have suggested improvements, it's very easy for them to send those to 
you, and you can make the fixes available immediately.  Once you are 
satisfied that it is stable, you can submit it to CRAN, and anyone using 
R can easily install it.


If you find the prospect of writing a package daunting, you shouldn't. 
It's actually quite easy, especially if you are using RStudio or ESS (or 
some other helpful front-end.)  Hadley Wickham's book
 is a pretty accessible description of a 
development strategy.  (It's not the only strategy, but lots of people 
use it.)


Duncan Murdoch

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[William Dunlap via R-devel]

Valgrind (without gctorture) reports memory misuse:

% R --debugger=valgrind --debugger-args="--leak-check=full --num-callers=18"
...
> x <- 1:20
> y <- rep(letters[1:5], length(x) / 5L)
> for (i in 1:1000) {
+   # x[y == 'a'] <- x[y == 'b']
+   x <- `[<-`(x, y == 'a', x[y == 'b'])
+   cat(i, '')
+ }
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
29 30 31 32 33 34 35 36 37 ==4711== Invalid read of size 1
==4711==at 0x501A40F: Rf_xlength (Rinlinedfuns.h:542)
==4711==by 0x501A40F: VectorAssign (subassign.c:658)
==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
==4711==by 0x5020100: do_subassign (subassign.c:1571)
==4711==by 0x4F66398: bcEval (eval.c:6795)
==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
==4711==by 0x4F7DA70: do_for (eval.c:2185)
==4711==by 0x4F7741C: Rf_eval (eval.c:691)
==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
==4711==by 0x40075A: main (Rmain.c:29)
==4711==  Address 0x19b3ab90 is 0 bytes inside a block of size 160,048
free'd
==4711==at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==4711==by 0x4FAFCB2: ReleaseLargeFreeVectors (memory.c:1055)
==4711==by 0x4FAFCB2: RunGenCollect (memory.c:1825)
==4711==by 0x4FAFCB2: R_gc_internal (memory.c:2998)
==4711==by 0x4FB166F: Rf_allocVector3 (memory.c:2682)
==4711==by 0x4FB2310: Rf_allocVector (Rinlinedfuns.h:577)
==4711==by 0x4FB2310: R_alloc (memory.c:2197)
==4711==by 0x5023F7A: logicalSubscript (subscript.c:575)
==4711==by 0x5026DA3: Rf_makeSubscript (subscript.c:994)
==4711==by 0x501A2F3: VectorAssign (subassign.c:656)
==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
==4711==by 0x5020100: do_subassign (subassign.c:1571)
==4711==by 0x4F66398: bcEval (eval.c:6795)
==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
==4711==by 0x4F7DA70: do_for (eval.c:2185)
==4711==by 0x4F7741C: Rf_eval (eval.c:691)
==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
==4711==by 0x40075A: main (Rmain.c:29)
==4711==  Block was alloc'd at
==4711==at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==4711==by 0x4FB1B04: Rf_allocVector3 (memory.c:2712)
==4711==by 0x5027574: Rf_allocVector (Rinlinedfuns.h:577)
==4711==by 0x5027574: Rf_ExtractSubset (subset.c:115)
==4711==by 0x502ADCD: VectorSubset (subset.c:198)
==4711==by 0x502ADCD: do_subset_dflt (subset.c:823)
==4711==by 0x502BE90: do_subset (subset.c:661)
==4711==by 0x4F7741C: Rf_eval (eval.c:691)
==4711==by 0x4F7BAC3: Rf_evalListKeepMissing (eval.c:2955)
==4711==by 0x50200CB: R_DispatchOrEvalSP (subassign.c:1535)
==4711==by 0x50200CB: do_subassign (subassign.c:1567)
==4711==by 0x4F66398: bcEval (eval.c:6795)
==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
==4711==by 0x4F7DA70: do_for (eval.c:2185)
==4711==by 0x4F7741C: Rf_eval (eval.c:691)
==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
==4711==by 0x40075A: main (Rmain.c:29)
==4711==
==4711== Invalid read of size 8
==4711==at 0x501A856: XLENGTH_EX (Rinlinedfuns.h:189)
==4711==by 0x501A856: Rf_xlength (Rinlinedfuns.h:554)
==4711==by 0x501A856: VectorAssign (subassign.c:658)
==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
==4711==by 0x5020100: do_subassign (subassign.c:1571)
==4711==by 0x4F66398: bcEval (eval.c:6795)
==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
==4711==by 0x4F7DA70: do_for (eval.c:2185)
==4711==by 0x4F7741C: Rf_eval (eval.c:691)
==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
==4711==by 0x40075A: main (Rmain.c:29)
==4711==  Address 0x19b3abb0 is 32 bytes inside a block of size 160,048
free'd
==4711==at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==4711==by 0x4FAFCB2: ReleaseLargeFreeVectors (memory.c:1055)
==4711==by 0x4FAFCB2: RunGenCollect (memory.c:1825)
==4711==by 0x4FAFCB2: R_gc_internal (memory.c:2998)
==4711==by 0x4FB166F: Rf_allocVector3 (memory.c:2682)
==4711==by 0x4FB2310: Rf_allocVector (Rinlinedfuns.h:577)
==4711==by 0x4FB2310: R_alloc (memory.c:2197)
==4711==by 0x5023F7A: logicalSubscript (subscript.c:575)
==4711==by 0x5026DA3: Rf_makeSubscript (subscript.c:994)
==4711==by 0x501A2F3: VectorAssign (subassign.c:656)
==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
==4711==by 0x5020100: do_subassign (subassign.c:1571)
==4711==by 0x4F66398: bcEval (eval.c:6795)
==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
==4711==by 0x4F7DA70:

[Rd] Problem with compiling OpenBLAS to work with R

[Erin Hodgess]

Hello!

I'm not sure if this is the right place to post this, so apologies
in advance if I'm not in the right list.

I downloaded the OpenBLAS and am following Avraham Adler's great
instructions.  However, when I run make, things go well to a certain point,
and then go bad:

make
[snip]

touch cygopenblas_haswellp-r0.3.5.a
make -j 1 -C test all
make[1]: Entering directory
'/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
gfortran -O2 -Wall -frecursive -m64 -mavx2   -o sblat1 sblat1.o
../cygopenblas_haswellp-r0.3.5.a  -L/usr/lib/gcc/x86_64-pc-msys/7.3.0
-L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib/../lib
-L/usr/lib/../lib
-L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib
-L/usr/lib/w32api  -lmsys-2.0
D:/msys64/usr/lib/../lib/libpthread.a(t-d001088.o):fake:(.text+0x2):
undefined reference to `__imp_pthread_mutex_destroy'
D:/msys64/usr/lib/../lib/libpthread.a(t-d001090.o):fake:(.text+0x2):
undefined reference to `__imp_pthread_mutex_init'
D:/msys64/usr/lib/../lib/libpthread.a(t-d001091.o):fake:(.text+0x2):
undefined reference to `__imp_pthread_mutex_lock'
D:/msys64/usr/lib/../lib/libpthread.a(t-d001094.o):fake:(.text+0x2):
undefined reference to `__imp_pthread_mutex_trylock'
D:/msys64/usr/lib/../lib/libpthread.a(t-d001095.o):fake:(.text+0x2):
undefined reference to `__imp_pthread_mutex_unlock'
collect2.exe: error: ld returned 1 exit status
make[1]: *** [Makefile:134: sblat1] Error 1
make[1]: Leaving directory
'/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
make: *** [Makefile:124: tests] Error 2


I think it has something to do with the threads/pthreads but am not sure
how to fix it.  Any suggestions much appreciated.

Thanks,
Sincerely,
Erin

Erin Hodgess, PhD
mailto: erinm.hodg...@gmail.com

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[Travers Ching]

On an azure centos VM, I can reproduce this bug which reports either:

 *** caught segfault ***
address 0x7006a, cause 'memory not mapped' (crash)

Or

incompatible types (from builtin to integer) in subassignment type fix
(no crash)

Like Gabriel, I could not reproduce the bug on a mac laptop.  Both R
versions 3.5.1.

Travers

On Wed, Feb 27, 2019 at 9:08 AM William Dunlap via R-devel
 wrote:
>
> Valgrind (without gctorture) reports memory misuse:
>
> % R --debugger=valgrind --debugger-args="--leak-check=full --num-callers=18"
> ...
> > x <- 1:20
> > y <- rep(letters[1:5], length(x) / 5L)
> > for (i in 1:1000) {
> +   # x[y == 'a'] <- x[y == 'b']
> +   x <- `[<-`(x, y == 'a', x[y == 'b'])
> +   cat(i, '')
> + }
> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
> 29 30 31 32 33 34 35 36 37 ==4711== Invalid read of size 1
> ==4711==at 0x501A40F: Rf_xlength (Rinlinedfuns.h:542)
> ==4711==by 0x501A40F: VectorAssign (subassign.c:658)
> ==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
> ==4711==by 0x5020100: do_subassign (subassign.c:1571)
> ==4711==by 0x4F66398: bcEval (eval.c:6795)
> ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> ==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
> ==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
> ==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
> ==4711==by 0x40075A: main (Rmain.c:29)
> ==4711==  Address 0x19b3ab90 is 0 bytes inside a block of size 160,048
> free'd
> ==4711==at 0x4C2ACBD: free (vg_replace_malloc.c:530)
> ==4711==by 0x4FAFCB2: ReleaseLargeFreeVectors (memory.c:1055)
> ==4711==by 0x4FAFCB2: RunGenCollect (memory.c:1825)
> ==4711==by 0x4FAFCB2: R_gc_internal (memory.c:2998)
> ==4711==by 0x4FB166F: Rf_allocVector3 (memory.c:2682)
> ==4711==by 0x4FB2310: Rf_allocVector (Rinlinedfuns.h:577)
> ==4711==by 0x4FB2310: R_alloc (memory.c:2197)
> ==4711==by 0x5023F7A: logicalSubscript (subscript.c:575)
> ==4711==by 0x5026DA3: Rf_makeSubscript (subscript.c:994)
> ==4711==by 0x501A2F3: VectorAssign (subassign.c:656)
> ==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
> ==4711==by 0x5020100: do_subassign (subassign.c:1571)
> ==4711==by 0x4F66398: bcEval (eval.c:6795)
> ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> ==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
> ==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
> ==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
> ==4711==by 0x40075A: main (Rmain.c:29)
> ==4711==  Block was alloc'd at
> ==4711==at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
> ==4711==by 0x4FB1B04: Rf_allocVector3 (memory.c:2712)
> ==4711==by 0x5027574: Rf_allocVector (Rinlinedfuns.h:577)
> ==4711==by 0x5027574: Rf_ExtractSubset (subset.c:115)
> ==4711==by 0x502ADCD: VectorSubset (subset.c:198)
> ==4711==by 0x502ADCD: do_subset_dflt (subset.c:823)
> ==4711==by 0x502BE90: do_subset (subset.c:661)
> ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> ==4711==by 0x4F7BAC3: Rf_evalListKeepMissing (eval.c:2955)
> ==4711==by 0x50200CB: R_DispatchOrEvalSP (subassign.c:1535)
> ==4711==by 0x50200CB: do_subassign (subassign.c:1567)
> ==4711==by 0x4F66398: bcEval (eval.c:6795)
> ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> ==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
> ==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
> ==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
> ==4711==by 0x40075A: main (Rmain.c:29)
> ==4711==
> ==4711== Invalid read of size 8
> ==4711==at 0x501A856: XLENGTH_EX (Rinlinedfuns.h:189)
> ==4711==by 0x501A856: Rf_xlength (Rinlinedfuns.h:554)
> ==4711==by 0x501A856: VectorAssign (subassign.c:658)
> ==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
> ==4711==by 0x5020100: do_subassign (subassign.c:1571)
> ==4711==by 0x4F66398: bcEval (eval.c:6795)
> ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> ==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
> ==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
> ==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
> ==4711==by 0x40075A: main (Rmain.c:29)
> ==4711==  Address 0x19b3abb0 is 32 bytes inside a block of size 160,048
> free'd
> ==4711==at 0x4C2ACBD: free (vg_replace_malloc.c:530)
> ==4711==by 0x4FAFCB2: ReleaseLargeFreeVectors (memory.c:1055)
> ==4711==by 0x4FAFCB2: RunGenCollect (memory.c:1825)
> ==4711==by 0x4FAFCB2: R_gc_internal (memory.c:2998)
> ==4711==by 0x4FB16

Re: [Rd] Intermittent crashes with inset `[<-` command

[Travers Ching]

Some testing:

Adding `gc()` inside the for loop prevented a crash for 10,000+
iterations, whereas adding `Sys.sleep(.2)` (which takes longer) did
not.  I couldn't wrap my head around the `vectorAssign` source code,
but I suspect it is a matter of an intermediate object not being
protected and being gc'ed.  Hope that helps someone

Travers


Travers

On Wed, Feb 27, 2019 at 11:48 AM Travers Ching  wrote:
>
> On an azure centos VM, I can reproduce this bug which reports either:
>
>  *** caught segfault ***
> address 0x7006a, cause 'memory not mapped' (crash)
>
> Or
>
> incompatible types (from builtin to integer) in subassignment type fix
> (no crash)
>
> Like Gabriel, I could not reproduce the bug on a mac laptop.  Both R
> versions 3.5.1.
>
> Travers
>
> On Wed, Feb 27, 2019 at 9:08 AM William Dunlap via R-devel
>  wrote:
> >
> > Valgrind (without gctorture) reports memory misuse:
> >
> > % R --debugger=valgrind --debugger-args="--leak-check=full --num-callers=18"
> > ...
> > > x <- 1:20
> > > y <- rep(letters[1:5], length(x) / 5L)
> > > for (i in 1:1000) {
> > +   # x[y == 'a'] <- x[y == 'b']
> > +   x <- `[<-`(x, y == 'a', x[y == 'b'])
> > +   cat(i, '')
> > + }
> > 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
> > 29 30 31 32 33 34 35 36 37 ==4711== Invalid read of size 1
> > ==4711==at 0x501A40F: Rf_xlength (Rinlinedfuns.h:542)
> > ==4711==by 0x501A40F: VectorAssign (subassign.c:658)
> > ==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
> > ==4711==by 0x5020100: do_subassign (subassign.c:1571)
> > ==4711==by 0x4F66398: bcEval (eval.c:6795)
> > ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> > ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> > ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> > ==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
> > ==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
> > ==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
> > ==4711==by 0x40075A: main (Rmain.c:29)
> > ==4711==  Address 0x19b3ab90 is 0 bytes inside a block of size 160,048
> > free'd
> > ==4711==at 0x4C2ACBD: free (vg_replace_malloc.c:530)
> > ==4711==by 0x4FAFCB2: ReleaseLargeFreeVectors (memory.c:1055)
> > ==4711==by 0x4FAFCB2: RunGenCollect (memory.c:1825)
> > ==4711==by 0x4FAFCB2: R_gc_internal (memory.c:2998)
> > ==4711==by 0x4FB166F: Rf_allocVector3 (memory.c:2682)
> > ==4711==by 0x4FB2310: Rf_allocVector (Rinlinedfuns.h:577)
> > ==4711==by 0x4FB2310: R_alloc (memory.c:2197)
> > ==4711==by 0x5023F7A: logicalSubscript (subscript.c:575)
> > ==4711==by 0x5026DA3: Rf_makeSubscript (subscript.c:994)
> > ==4711==by 0x501A2F3: VectorAssign (subassign.c:656)
> > ==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
> > ==4711==by 0x5020100: do_subassign (subassign.c:1571)
> > ==4711==by 0x4F66398: bcEval (eval.c:6795)
> > ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> > ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> > ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> > ==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
> > ==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
> > ==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
> > ==4711==by 0x40075A: main (Rmain.c:29)
> > ==4711==  Block was alloc'd at
> > ==4711==at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
> > ==4711==by 0x4FB1B04: Rf_allocVector3 (memory.c:2712)
> > ==4711==by 0x5027574: Rf_allocVector (Rinlinedfuns.h:577)
> > ==4711==by 0x5027574: Rf_ExtractSubset (subset.c:115)
> > ==4711==by 0x502ADCD: VectorSubset (subset.c:198)
> > ==4711==by 0x502ADCD: do_subset_dflt (subset.c:823)
> > ==4711==by 0x502BE90: do_subset (subset.c:661)
> > ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> > ==4711==by 0x4F7BAC3: Rf_evalListKeepMissing (eval.c:2955)
> > ==4711==by 0x50200CB: R_DispatchOrEvalSP (subassign.c:1535)
> > ==4711==by 0x50200CB: do_subassign (subassign.c:1567)
> > ==4711==by 0x4F66398: bcEval (eval.c:6795)
> > ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> > ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> > ==4711==by 0x4F7741C: Rf_eval (eval.c:691)
> > ==4711==by 0x4FA7181: Rf_ReplIteration (main.c:258)
> > ==4711==by 0x4FA7570: R_ReplConsole (main.c:308)
> > ==4711==by 0x4FA760E: run_Rmainloop (main.c:1082)
> > ==4711==by 0x40075A: main (Rmain.c:29)
> > ==4711==
> > ==4711== Invalid read of size 8
> > ==4711==at 0x501A856: XLENGTH_EX (Rinlinedfuns.h:189)
> > ==4711==by 0x501A856: Rf_xlength (Rinlinedfuns.h:554)
> > ==4711==by 0x501A856: VectorAssign (subassign.c:658)
> > ==4711==by 0x501CDFE: do_subassign_dflt (subassign.c:1641)
> > ==4711==by 0x5020100: do_subassign (subassign.c:1571)
> > ==4711==by 0x4F66398: bcEval (eval.c:6795)
> > ==4711==by 0x4F7D86D: R_compileAndExecute (eval.c:1407)
> > ==4711==by 0x4F7DA70: do_for (eval.c:2185)
> > ==4711==   

Re: [Rd] Intermittent crashes with inset `[<-` command

[Tierney, Luke]

Thanks for the report. Should be fixed shortly.

Best,

luke

On Tue, 26 Feb 2019, Brian Montgomery via R-devel wrote:

> The following code crashes after about 300 iterations on my 
> x86_64-w64-mingw32 machine on R 3.5.2 --vanilla.  
> Others have duplicated this (see 
> https://github.com/tidyverse/magrittr/issues/190 if necessary), but I don't 
> know how machine/OS-dependent it may be.  
> If it doesn't crash for you, please try increasing the length of the x vector.
>
> Substituting the commented-out line for the one below it works correctly 
> (prints out 1:1000 and ends normally) every time.
>
> x <- 1:20
> y <- rep(letters[1:5], length(x) / 5L)
> for (i in 1:1000) {
>   # x[y == 'a'] <- x[y == 'b']
>   x <- `[<-`(x, y == 'a', x[y == 'b'])
>   cat(i, '')
> }
> cat('\n')
>
> The point of using this syntax is to make it work better with pipes, but the 
> errors occur without pipes or magrittr.
>
> Thank you for your help!
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa  Phone: 319-335-3386
Department of Statistics andFax:   319-335-3017
Actuarial Science
241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[Tierney, Luke]

On Tue, 26 Feb 2019, Brian Montgomery via R-devel wrote:

> The following code crashes after about 300 iterations on my 
> x86_64-w64-mingw32 machine on R 3.5.2 --vanilla.  
> Others have duplicated this (see 
> https://github.com/tidyverse/magrittr/issues/190 if necessary), but I don't 
> know how machine/OS-dependent it may be.  
> If it doesn't crash for you, please try increasing the length of the x vector.
>
> Substituting the commented-out line for the one below it works correctly 
> (prints out 1:1000 and ends normally) every time.
>
> x <- 1:20
> y <- rep(letters[1:5], length(x) / 5L)
> for (i in 1:1000) {
>   # x[y == 'a'] <- x[y == 'b']
>   x <- `[<-`(x, y == 'a', x[y == 'b'])
>   cat(i, '')
> }
> cat('\n')
>
> The point of using this syntax is to make it work better with pipes, but the 
> errors occur without pipes or magrittr.

Calling replacement functions this way is a Really Bad Idea. Some
assume they are being called properly and will end up mutating data
they should not when called this way.

Best,

luke

>
> Thank you for your help!
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa  Phone: 319-335-3386
Department of Statistics andFax:   319-335-3017
Actuarial Science
241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Intermittent crashes with inset `[<-` command

[Tierney, Luke]

Fixed in R-devel and R-patched.

Best,

luke

On Wed, 27 Feb 2019, Tierney, Luke wrote:

> Thanks for the report. Should be fixed shortly.
>
> Best,
>
> luke
>
> On Tue, 26 Feb 2019, Brian Montgomery via R-devel wrote:
>
>> The following code crashes after about 300 iterations on my 
>> x86_64-w64-mingw32 machine on R 3.5.2 --vanilla.  
>> Others have duplicated this (see 
>> https://github.com/tidyverse/magrittr/issues/190 if necessary), but I don't 
>> know how machine/OS-dependent it may be.  
>> If it doesn't crash for you, please try increasing the length of the x 
>> vector.
>>
>> Substituting the commented-out line for the one below it works correctly 
>> (prints out 1:1000 and ends normally) every time.
>>
>> x <- 1:20
>> y <- rep(letters[1:5], length(x) / 5L)
>> for (i in 1:1000) {
>>   # x[y == 'a'] <- x[y == 'b']
>>   x <- `[<-`(x, y == 'a', x[y == 'b'])
>>   cat(i, '')
>> }
>> cat('\n')
>>
>> The point of using this syntax is to make it work better with pipes, but the 
>> errors occur without pipes or magrittr.
>>
>> Thank you for your help!
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
>

-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa  Phone: 319-335-3386
Department of Statistics andFax:   319-335-3017
Actuarial Science
241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] stopifnot

[Suharto Anggono Suharto Anggono via R-devel]

My points:
- The 'withCallingHandlers' construct that is used in current 'stopifnot' code 
has no effect. Without it, the warning message is the same. The overridden 
warning is not raised. The original warning stays.
- Overriding call in error and warning to 'cl.i' doesn't always give better 
outcome. The original call may be "narrower" than 'cl.i'.

I have found these examples.
identity(is.na(log()))
identity(is.na(log("a")))

Error message from the first contains full call. Error message from the second 
doesn't.

So, how about being "natural", not using 'withCallingHandlers' and 'tryCatch' 
in 'stopifnot'?

Another thing: currently,
stopifnot(exprs=TRUE)
fails.

A patch:
--- stop.R  2019-02-27 16:15:45.324167577 +
+++ stop_new.R  2019-02-27 16:22:15.936203541 +
@@ -1,7 +1,7 @@
 #  File src/library/base/R/stop.R
 #  Part of the R package, https://www.R-project.org
 #
-#  Copyright (C) 1995-2018 The R Core Team
+#  Copyright (C) 1995-2019 The R Core Team
 #
 #  This program is free software; you can redistribute it and/or modify
 #  it under the terms of the GNU General Public License as published by
@@ -33,25 +33,27 @@
 
 stopifnot <- function(..., exprs, local = TRUE)
 {
+n <- ...length()
 missE <- missing(exprs)
-cl <-
if(missE) {  ## use '...' instead of exprs
-   match.call(expand.dots=FALSE)$...
} else {
-   if(...length())
+   if(n)
stop("Must use 'exprs' or unnamed expressions, but not both")
envir <- if (isTRUE(local)) parent.frame()
 else if(isFALSE(local)) .GlobalEnv
 else if (is.environment(local)) local
 else stop("'local' must be TRUE, FALSE or an environment")
exprs <- substitute(exprs) # protect from evaluation
-   E1 <- exprs[[1]]
+   E1 <- if(is.call(exprs)) exprs[[1]]
+   cl <-
if(identical(quote(`{`), E1)) # { ... }
-   do.call(expression, as.list(exprs[-1]))
+   exprs[-1]
else if(identical(quote(expression), E1))
eval(exprs, envir=envir)
else
as.expression(exprs) # or fail ..
+   if(!is.null(names(cl))) names(cl) <- NULL
+   return(eval(as.call(c(sys.call()[[1]], as.list(cl))), envir=envir))
}
 Dparse <- function(call, cutoff = 60L) {
ch <- deparse(call, width.cutoff = cutoff)
@@ -62,14 +64,10 @@
 abbrev <- function(ae, n = 3L)
paste(c(head(ae, n), if(length(ae) > n) ""), collapse="\n  ")
 ##
-for (i in seq_along(cl)) {
-   cl.i <- cl[[i]]
-   ## r <- eval(cl.i, ..)   # with correct warn/err messages:
-   r <- withCallingHandlers(
-   tryCatch(if(missE) ...elt(i) else eval(cl.i, envir=envir),
-error = function(e) { e$call <- cl.i; stop(e) }),
-   warning = function(w) { w$call <- cl.i; w })
+for (i in seq_len(n)) {
+   r <- ...elt(i)
if (!(is.logical(r) && !anyNA(r) && all(r))) {
+   cl.i <- match.call(expand.dots=FALSE)$...[[i]]
msg <- ## special case for decently written 'all.equal(*)':
if(is.call(cl.i) && identical(cl.i[[1]], quote(all.equal)) &&
   (is.null(ni <- names(cl.i)) || length(cl.i) == 3L ||
@@ -84,7 +82,11 @@
 "%s are not all TRUE"),
Dparse(cl.i))
 
-   stop(simpleError(msg, call = sys.call(-1)))
+   p <- sys.parent()
+   if(p && identical(sys.function(p), stopifnot) &&
+  !eval(expression(missE), p)) # originally stopifnot(exprs=*)
+   p <- sys.parent(2)
+   stop(simpleError(msg, call = if(p) sys.call(p)))
}
 }
 invisible()


On Wed, 27/2/19, Martin Maechler  wrote:

 Subject: Re: [Rd] stopifnot

 Cc: r-devel@r-project.org
 Date: Wednesday, 27 February, 2019, 5:36 PM

> Suharto Anggono Suharto Anggono via R-devel 
>    on Sun, 24 Feb 2019 14:22:48 + writes:

    >> From https://github.com/HenrikBengtsson/Wishlist-for-R/issues/70 :
    > ... and follow up note from 2018-03-15: Ouch... in R-devel, stopifnot() 
has become yet 4-5 times slower;

    > ...
    > which is due to a complete rewrite using tryCatch() and 
withCallingHandlers().


    >> From https://stat.ethz.ch/pipermail/r-devel/2017-May/074256.html , it 
seems that 'tryCatch' was used to avoid the following example from giving error 
message with 'eval' call and 'withCallingHandlers' was meant to handle similar 
case for warning.
    > tst <- function(y) { stopifnot(is.numeric(y)); y+ 1 }
    > try(tst())

    > However,
    > withCallingHandlers(,
    > warning = function(w) { w$call <- cl.i; w })
    > actally has no effect. In current code of function 'stopifnot', 'eval' is 
used only in handling stopifnot(exprs=) . The warning message from
    > stopifnot(ex

Re: [Rd] Problem with compiling OpenBLAS to work with R

[Kenny Bell]

This person has had apparent success - you could follow what they did or
just download their product (with appropriate caution downloading a random
.exe).

https://github.com/thequackdaddy/R-OpenBLAS

On Thu, Feb 28, 2019 at 6:28 AM Erin Hodgess 
wrote:

> Hello!
>
> I'm not sure if this is the right place to post this, so apologies
> in advance if I'm not in the right list.
>
> I downloaded the OpenBLAS and am following Avraham Adler's great
> instructions.  However, when I run make, things go well to a certain point,
> and then go bad:
>
> make
> [snip]
>
> touch cygopenblas_haswellp-r0.3.5.a
> make -j 1 -C test all
> make[1]: Entering directory
> '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> gfortran -O2 -Wall -frecursive -m64 -mavx2   -o sblat1 sblat1.o
> ../cygopenblas_haswellp-r0.3.5.a  -L/usr/lib/gcc/x86_64-pc-msys/7.3.0
> -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib/../lib
> -L/usr/lib/../lib
> -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib
> -L/usr/lib/w32api  -lmsys-2.0
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001088.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_destroy'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001090.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_init'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001091.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_lock'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001094.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_trylock'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001095.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_unlock'
> collect2.exe: error: ld returned 1 exit status
> make[1]: *** [Makefile:134: sblat1] Error 1
> make[1]: Leaving directory
> '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> make: *** [Makefile:124: tests] Error 2
>
>
> I think it has something to do with the threads/pthreads but am not sure
> how to fix it.  Any suggestions much appreciated.
>
> Thanks,
> Sincerely,
> Erin
>
> Erin Hodgess, PhD
> mailto: erinm.hodg...@gmail.com
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Problem with compiling OpenBLAS to work with R

[Avraham Adler]

I believe that repo just follows the directions on my blog. Without seeing
Dr. Hodges’s code, my initial concern is the many references to Cygwin. My
method specifically does not use Cygwin but MSYS2 and Mingw64/Rtools35.
That will likely change to solely Rtools40 once R3.6 is released due to the
Msys system being built in to it.

There may be some library conflicts between Cygwin and msys2/mingw64. If
possible, my suggestion would be uninstall everything and then just install
msys2 (and add in make after you to the first msys update) and rtools35.
Then there should be no conflicting libraries.

Thanks,

Avi

On Thu, Feb 28, 2019 at 12:11 AM Kenny Bell  wrote:

> This person has had apparent success - you could follow what they did or
> just download their product (with appropriate caution downloading a random
> .exe).
>
> https://github.com/thequackdaddy/R-OpenBLAS
>
> On Thu, Feb 28, 2019 at 6:28 AM Erin Hodgess 
> wrote:
>
> > Hello!
> >
> > I'm not sure if this is the right place to post this, so apologies
> > in advance if I'm not in the right list.
> >
> > I downloaded the OpenBLAS and am following Avraham Adler's great
> > instructions.  However, when I run make, things go well to a certain
> point,
> > and then go bad:
> >
> > make
> > [snip]
> >
> > touch cygopenblas_haswellp-r0.3.5.a
> > make -j 1 -C test all
> > make[1]: Entering directory
> > '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> > gfortran -O2 -Wall -frecursive -m64 -mavx2   -o sblat1 sblat1.o
> > ../cygopenblas_haswellp-r0.3.5.a  -L/usr/lib/gcc/x86_64-pc-msys/7.3.0
> > -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib/../lib
> > -L/usr/lib/../lib
> > -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib
> > -L/usr/lib/w32api  -lmsys-2.0
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001088.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_destroy'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001090.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_init'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001091.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_lock'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001094.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_trylock'
> > D:/msys64/usr/lib/../lib/libpthread.a(t-d001095.o):fake:(.text+0x2):
> > undefined reference to `__imp_pthread_mutex_unlock'
> > collect2.exe: error: ld returned 1 exit status
> > make[1]: *** [Makefile:134: sblat1] Error 1
> > make[1]: Leaving directory
> > '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> > make: *** [Makefile:124: tests] Error 2
> >
> >
> > I think it has something to do with the threads/pthreads but am not sure
> > how to fix it.  Any suggestions much appreciated.
> >
> > Thanks,
> > Sincerely,
> > Erin
> >
> > Erin Hodgess, PhD
> > mailto: erinm.hodg...@gmail.com
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Sent from Gmail Mobile

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel