Re: [Rd] trivial typo in src/library/base/man/timezone.Rd
On 01/03/2014 20:47, Ben Bolker wrote: From SVN r65092: line 111: "Note that that the abbreviations have changed over the years" (duplicated "that") Ben Bolker __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel This is work in progress: see e.g. http://cran.r-project.org/bin/windows/base/rdevel.html . Especially where there are multiple versions to be checked, it can take a few days to finish things like this, and that was already re-written in the working version. -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] reference classes, LAZY_DUPLICATE_OK, and external pointers
We (the lme4 authors) are having a problem with doing a proper deep
copy of a reference class object in recent versions of R-devel with
the LAZY_DUPLICATE_OK flag in src/main/bind.c enabled.
Apologies in advance for any improper terminology.
TL;DR Is there an elegant way to force non-lazy/deep copying in our
case? Is anyone else using reference classes with a field that is an
external pointer?
This is how copying of reference classes works in a normal
situation:
library(data.table) ## for address() function
setRefClass("defaultRC",fields="theta")
d1 <- new("defaultRC")
d1$theta <- 1
address(d1$theta) ## "0xb70"
d2 <- d1$copy()
address(d2$theta) ## same as above
d2$theta <- 2
address(d2$theta) ## now modified, by magic
d1$theta ## unmodified
The extra complication in our case is that many of the objects within
our reference class are actually accessed via an external pointer,
which is initialized when necessary -- details are copied below for
those who want them, or you can see the code at
https://github.com/lme4/lme4
The problem is that this sneaky way of copying the object's contents
doesn't trigger R's (new) rules for recognizing that a non-lazy copy
should be made.
library(lme4)
fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
pp <- fm1@pp
pp$theta ## [1] 0.96673279 0.01516906 0.23090960
address(pp$theta) ## something
pp$Ptr ##
xpp <- pp$copy() ## default is deep copy
xpp$Ptr ##
address(xpp$theta) ## same as above
xpp$setTheta(c(0,0,0)) ## referenced through Ptr field
xpp$Ptr ## now set to non-nil
fm1@pp$theta ## changes to (0,0,0). oops.
So apparently when the xpp$theta object is copied into the external
pointer, a reference/lazy copy is made. (xpp$theta itself is
read-only, so I can't do the assignment that way)
I can hack around this in a very ugly way by doing a trivial
modification when assigning inside the copy method:
assign("theta",get("theta",envir=selfEnv)+0, envir=vEnv)
... but (a) this is very ugly and (b) it seems very unsafe --
as R gets smarter it should start to recognize trivial changes
like x+0 and x*1 and *not* copy in these cases ...
Method details:
## from R/AllClass.R, merPredD RC definition
ptr = function() {
'returns the external pointer, regenerating if necessary'
if (length(theta)) {
if (.Call(isNullExtPtr, Ptr)) initializePtr()
}
Ptr
},
## ditto
initializePtr = function() {
Ptr <<- .Call(merPredDCreate, as(X, "matrix"), Lambdat,
LamtUt, Lind, RZX, Ut, Utr, V, VtV, Vtr,
Xwts, Zt, beta0, delb, delu, theta, u0)
...
}
merPredDCreate in turn just copies the relevant bits into a new C++
class object:
/* see src/external.cpp */
SEXP merPredDCreate(SEXP Xs, SEXP Lambdat, SEXP LamtUt, SEXP Lind,
SEXP RZX, SEXP Ut, SEXP Utr, SEXP V, SEXP VtV,
SEXP Vtr, SEXP Xwts, SEXP Zt, SEXP beta0,
SEXP delb, SEXP delu, SEXP theta, SEXP u0) {
BEGIN_RCPP;
merPredD *ans = new merPredD(Xs, Lambdat, LamtUt, Lind, RZX,
Ut, Utr, V, VtV,
Vtr, Xwts, Zt, beta0, delb, delu,
theta, u0);
return wrap(XPtr(ans, true));
END_RCPP;
}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] reference classes, LAZY_DUPLICATE_OK, and external pointers
Ben,
On Mar 2, 2014, at 7:38 PM, Ben Bolker wrote:
> We (the lme4 authors) are having a problem with doing a proper deep
> copy of a reference class object in recent versions of R-devel with
> the LAZY_DUPLICATE_OK flag in src/main/bind.c enabled.
>
> Apologies in advance for any improper terminology.
>
> TL;DR Is there an elegant way to force non-lazy/deep copying in our
> case? Is anyone else using reference classes with a field that is an
> external pointer?
>
> This is how copying of reference classes works in a normal
> situation:
>
> library(data.table) ## for address() function
> setRefClass("defaultRC",fields="theta")
> d1 <- new("defaultRC")
> d1$theta <- 1
> address(d1$theta) ## "0xb70"
> d2 <- d1$copy()
> address(d2$theta) ## same as above
> d2$theta <- 2
> address(d2$theta) ## now modified, by magic
> d1$theta ## unmodified
>
> The extra complication in our case is that many of the objects within
> our reference class are actually accessed via an external pointer,
> which is initialized when necessary -- details are copied below for
> those who want them, or you can see the code at
> https://github.com/lme4/lme4
>
> The problem is that this sneaky way of copying the object's contents
> doesn't trigger R's (new) rules for recognizing that a non-lazy copy
> should be made.
>
This is not R's decision - AFAICS your code is incorrectly assuming that there
is no other reference where there is no such guarantee. Your code that assigns
into the external pointer has to make that decision - it's not R's to make
since you are taking the full responsibility for external pointers by
circumventing R's handing. External pointers had always had reference
semantics. Note that this is not new - you had to inspect the NAMED bits and
call duplicate() yourself to guarantee a copy even in previous R versions. It
just so happened that bugs of not doing so were often masked by R being more
conservative such that in some circumstanced there were enough references to
function arguments that R would defensively create a new copy.
So, the same applies as it did before - if you store something that you want to
be mutable in C/C++ you have to check the references and call duplicate() if
you don't own the only reference.
Cheers,
Simon
> library(lme4)
> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
> pp <- fm1@pp
> pp$theta ## [1] 0.96673279 0.01516906 0.23090960
> address(pp$theta) ## something
> pp$Ptr ##
> xpp <- pp$copy() ## default is deep copy
> xpp$Ptr ##
> address(xpp$theta) ## same as above
> xpp$setTheta(c(0,0,0)) ## referenced through Ptr field
> xpp$Ptr ## now set to non-nil
> fm1@pp$theta ## changes to (0,0,0). oops.
>
> So apparently when the xpp$theta object is copied into the external
> pointer, a reference/lazy copy is made. (xpp$theta itself is
> read-only, so I can't do the assignment that way)
>
> I can hack around this in a very ugly way by doing a trivial
> modification when assigning inside the copy method:
>
> assign("theta",get("theta",envir=selfEnv)+0, envir=vEnv)
>
> ... but (a) this is very ugly and (b) it seems very unsafe --
> as R gets smarter it should start to recognize trivial changes
> like x+0 and x*1 and *not* copy in these cases ...
>
> Method details:
>
> ## from R/AllClass.R, merPredD RC definition
>
> ptr = function() {
> 'returns the external pointer, regenerating if necessary'
> if (length(theta)) {
>if (.Call(isNullExtPtr, Ptr)) initializePtr()
> }
> Ptr
> },
>
> ## ditto
>
> initializePtr = function() {
>Ptr <<- .Call(merPredDCreate, as(X, "matrix"), Lambdat,
> LamtUt, Lind, RZX, Ut, Utr, V, VtV, Vtr,
> Xwts, Zt, beta0, delb, delu, theta, u0)
> ...
> }
>
> merPredDCreate in turn just copies the relevant bits into a new C++
> class object:
>
> /* see src/external.cpp */
>
>SEXP merPredDCreate(SEXP Xs, SEXP Lambdat, SEXP LamtUt, SEXP Lind,
>SEXP RZX, SEXP Ut, SEXP Utr, SEXP V, SEXP VtV,
>SEXP Vtr, SEXP Xwts, SEXP Zt, SEXP beta0,
>SEXP delb, SEXP delu, SEXP theta, SEXP u0) {
>BEGIN_RCPP;
>merPredD *ans = new merPredD(Xs, Lambdat, LamtUt, Lind, RZX,
> Ut, Utr, V, VtV,
> Vtr, Xwts, Zt, beta0, delb, delu,
> theta, u0);
>return wrap(XPtr(ans, true));
>END_RCPP;
>}
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] reference classes, LAZY_DUPLICATE_OK, and external pointers
On 14-03-02 08:05 PM, Simon Urbanek wrote:
> Ben,
>
> On Mar 2, 2014, at 7:38 PM, Ben Bolker wrote:
>
>> We (the lme4 authors) are having a problem with doing a proper
>> deep copy of a reference class object in recent versions of R-devel
>> with the LAZY_DUPLICATE_OK flag in src/main/bind.c enabled.
>>
>> Apologies in advance for any improper terminology.
>>
>> TL;DR Is there an elegant way to force non-lazy/deep copying in
>> our case? Is anyone else using reference classes with a field that
>> is an external pointer?
>>
>> This is how copying of reference classes works in a normal
>> situation:
>>
>> library(data.table) ## for address() function
>> setRefClass("defaultRC",fields="theta") d1 <- new("defaultRC")
>> d1$theta <- 1 address(d1$theta) ## "0xb70" d2 <- d1$copy()
>> address(d2$theta) ## same as above d2$theta <- 2 address(d2$theta)
>> ## now modified, by magic d1$theta ## unmodified
>>
>> The extra complication in our case is that many of the objects
>> within our reference class are actually accessed via an external
>> pointer, which is initialized when necessary -- details are copied
>> below for those who want them, or you can see the code at
>> https://github.com/lme4/lme4
>>
>> The problem is that this sneaky way of copying the object's
>> contents doesn't trigger R's (new) rules for recognizing that a
>> non-lazy copy should be made.
>>
>
> This is not R's decision - AFAICS your code is incorrectly assuming
> that there is no other reference where there is no such guarantee.
> Your code that assigns into the external pointer has to make that
> decision - it's not R's to make since you are taking the full
> responsibility for external pointers by circumventing R's handing.
> External pointers had always had reference semantics. Note that this
> is not new - you had to inspect the NAMED bits and call duplicate()
> yourself to guarantee a copy even in previous R versions. It just so
> happened that bugs of not doing so were often masked by R being more
> conservative such that in some circumstanced there were enough
> references to function arguments that R would defensively create a
> new copy.
>
> So, the same applies as it did before - if you store something that
> you want to be mutable in C/C++ you have to check the references and
> call duplicate() if you don't own the only reference.
>
> Cheers, Simon
Thanks, that's extremely useful.
Ben
>
>
>
>> library(lme4) fm1 <- lmer(Reaction ~ Days + (Days|Subject),
>> sleepstudy) pp <- fm1@pp pp$theta ## [1] 0.96673279 0.01516906
>> 0.23090960 address(pp$theta) ## something pp$Ptr ##
>> xpp <- pp$copy() ## default is deep copy xpp$Ptr ## > (nil)> address(xpp$theta) ## same as above xpp$setTheta(c(0,0,0))
>> ## referenced through Ptr field xpp$Ptr ## now set to non-nil
>> fm1@pp$theta ## changes to (0,0,0). oops.
>>
>> So apparently when the xpp$theta object is copied into the
>> external pointer, a reference/lazy copy is made. (xpp$theta
>> itself is read-only, so I can't do the assignment that way)
>>
>> I can hack around this in a very ugly way by doing a trivial
>> modification when assigning inside the copy method:
>>
>> assign("theta",get("theta",envir=selfEnv)+0, envir=vEnv)
>>
>> ... but (a) this is very ugly and (b) it seems very unsafe -- as R
>> gets smarter it should start to recognize trivial changes like x+0
>> and x*1 and *not* copy in these cases ...
>>
>> Method details:
>>
>> ## from R/AllClass.R, merPredD RC definition
>>
>> ptr = function() { 'returns the external pointer,
>> regenerating if necessary' if (length(theta)) { if
>> (.Call(isNullExtPtr, Ptr)) initializePtr() } Ptr },
>>
>> ## ditto
>>
>> initializePtr = function() { Ptr <<- .Call(merPredDCreate, as(X,
>> "matrix"), Lambdat, LamtUt, Lind, RZX, Ut, Utr, V, VtV, Vtr, Xwts,
>> Zt, beta0, delb, delu, theta, u0) ... }
>>
>> merPredDCreate in turn just copies the relevant bits into a new
>> C++ class object:
>>
>> /* see src/external.cpp */
>>
>> SEXP merPredDCreate(SEXP Xs, SEXP Lambdat, SEXP LamtUt, SEXP Lind,
>> SEXP RZX, SEXP Ut, SEXP Utr, SEXP V, SEXP VtV, SEXP Vtr, SEXP Xwts,
>> SEXP Zt, SEXP beta0, SEXP delb, SEXP delu, SEXP theta, SEXP u0) {
>> BEGIN_RCPP; merPredD *ans = new merPredD(Xs, Lambdat, LamtUt, Lind,
>> RZX, Ut, Utr, V, VtV, Vtr, Xwts, Zt, beta0, delb, delu, theta,
>> u0); return wrap(XPtr(ans, true)); END_RCPP; }
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
