[Rd] Finalizer execution order question
Given an externalptr object 'pool' which protects an R object 'prot':
# SEXP prot = (a dynamically updated list with handles)
SEXP pool = R_MakeExternalPtr(p, R_NilValue, prot);
R_RegisterCFinalizerEx(pool, fin_pool, TRUE);
WRE explains that 'prot' remains in existence as long as 'pool' is
around. Does this also mean 'prot' still exists when the finalizer of
'pool' gets executed?
Long story: I am running into an issue with the next generation of the
curl package which uses dynamic pools of (many) http request handles.
Both the pool and the handles are externalptr objects with finalizers
that clean up after themselves.
When the pool goes out of scope, its finalizer has to loop over
pending handles to release them. However, to my surprise, the
finalizers of the handles get executed *before* the finalizer of
'pool'. Therefore the handles have destroyed themselves before I can
properly take hem out of the pool. This is exactly what I was trying
to prevent by putting the handles in a 'prot' list.
I wrote a simple package [1] that illustrates this issue. To run the example:
devtools::install_github("jeroenooms/test")
library(test)
pool <- make_pool()
show_handles(pool)
rm(pool)
gc()
What this example is supposed to illustrate is that even though 'prot'
gets protected from GC while the externalptr is around, the finalizers
in 'prot' have already executed when the externalptr gets finalized.
What is even stranger (to me) is that the SEXPs in 'prot' still seem
in tact during the finalizer of externalptr (ASAN is not giving
use-after-free warnings either). It's just that their finalizers have
already executed. So that leaves the pool finalizer in the odd
position of seeing only the post-mortem SEXP contents of the handles.
This does provide a workaround in my case, but can we rely on this?
So my question: is this expected behavior? What would be an
alternative approach to ensure that handles stay protected until the
pool has been cleaned and finalized (rather than running all
finalizers in the order they were registered)?
[1] https://github.com/jeroenooms/test
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Finalizer execution order question
On Thu, 15 Sep 2016, Jeroen Ooms wrote:
Given an externalptr object 'pool' which protects an R object 'prot':
# SEXP prot = (a dynamically updated list with handles)
SEXP pool = R_MakeExternalPtr(p, R_NilValue, prot);
R_RegisterCFinalizerEx(pool, fin_pool, TRUE);
WRE explains that 'prot' remains in existence as long as 'pool' is
around. Does this also mean 'prot' still exists when the finalizer of
'pool' gets executed?
prot is still reachable and valid. If it also has a finalizer on it
then that finalizer might have been run before the one on prot -- you
should not assume anything about the order in which finalizers are run.
Long story: I am running into an issue with the next generation of the
curl package which uses dynamic pools of (many) http request handles.
Both the pool and the handles are externalptr objects with finalizers
that clean up after themselves.
When the pool goes out of scope, its finalizer has to loop over
pending handles to release them. However, to my surprise, the
finalizers of the handles get executed *before* the finalizer of
'pool'. Therefore the handles have destroyed themselves before I can
properly take hem out of the pool. This is exactly what I was trying
to prevent by putting the handles in a 'prot' list.
If your handles are always in a pool, then put a finalizer on the pool
but not on the handles?
Lists of weak references might also be useful to consider for this
sort of thing. I've used those to make sure finalizers are run before
a package DLL is inloaded (since running them later would not be a
good idea).
Best,
luke
I wrote a simple package [1] that illustrates this issue. To run the example:
devtools::install_github("jeroenooms/test")
library(test)
pool <- make_pool()
show_handles(pool)
rm(pool)
gc()
What this example is supposed to illustrate is that even though 'prot'
gets protected from GC while the externalptr is around, the finalizers
in 'prot' have already executed when the externalptr gets finalized.
What is even stranger (to me) is that the SEXPs in 'prot' still seem
in tact during the finalizer of externalptr (ASAN is not giving
use-after-free warnings either). It's just that their finalizers have
already executed. So that leaves the pool finalizer in the odd
position of seeing only the post-mortem SEXP contents of the handles.
This does provide a workaround in my case, but can we rely on this?
So my question: is this expected behavior? What would be an
alternative approach to ensure that handles stay protected until the
pool has been cleaned and finalized (rather than running all
finalizers in the order they were registered)?
[1] https://github.com/jeroenooms/test
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] row names of 'rowsum()'
'rowsum()' seems to add row names to the resulting matrix, corresponding to the respective 'group' values. This is very handy, but it is not documented. Should the documentation mention it so it could be relied upon as part of API? Cheers, Ott -- Ott Toomet Visiting Researcher School of Information Mary Gates Hall, Suite 310 University of Washington Seattle, WA 98195 [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Time zone issues when compiling R
I've been trying to build R 3.3.1 inside of a Nix environment on a
Ubuntu 16.04 machine. It builds, but then it fails a regression test
related to time zones, and I hope that someone could help me debug the
problem.
The failing test is in tests/reg-tests-rc.R
(https://github.com/wch/r-source/blob/c3fe9cd4/tests/reg-tests-1c.R#L1577-L1587):
## format.POSIXlt() of Jan.1 if 1941 or '42 is involved:
tJan1 <- function(n1, n2)
strptime(paste0(n1:n2,"/01/01"), "%Y/%m/%d", tz="CET")
wDSTJan1 <- function(n1, n2)
which("CEST" == sub(".* ", '', format(tJan1(n1,n2), usetz=TRUE)))
(w8 <- wDSTJan1(1801, 2300))
(w9 <- wDSTJan1(1901, 2300))
stopifnot(identical(w8, 141:142),# exactly 1941:1942 had CEST on Jan.1
identical(w9, 41: 42))
## for R-devel Jan.2016 to Mar.14 -- *AND* for R 3.2.4 -- the above gave
## integer(0) and c(41:42, 99:100, ..., 389:390) respectively
The resulting output is:
> (w8 <- wDSTJan1(1801, 2300))
integer(0)
> (w9 <- wDSTJan1(1901, 2300))
integer(0)
> stopifnot(identical(w8, 141:142),# exactly 1941:1942 had CEST on Jan.1
+ identical(w9, 41: 42))
Error: identical(w8, 141:142) is not TRUE
Execution halted
In this build of R, I get the following:
> strptime(paste0(1940:1945,"/01/01"), "%Y/%m/%d", tz="CET")
[1] "1940-01-01 CET" "1941-01-01 CET" "1942-01-01 CET" "1943-01-01 CET"
[5] "1944-01-01 CET" "1945-01-01 CET"
However, when I run the same code in R 3.3.1 installed via the .deb
packages on cran.r-project.org, I get a different result: years 1941
and 1942 have the CEST time zone. This is what the tests above were
expecting.
> strptime(paste0(1940:1945,"/01/01"), "%Y/%m/%d", tz="CET")
[1] "1940-01-01 CET" "1941-01-01 CEST" "1942-01-01 CEST" "1943-01-01 CET"
[5] "1944-01-01 CET" "1945-01-01 CET"
I'm not sure where to start looking to fix this problem, and I'd
appreciate any pointers.
For reference, the test was introduced in:
https://github.com/wch/r-source/commit/55cdf88dv
And that commit's message says that it fixed a bug introduced by:
https://github.com/wch/r-source/commit/2e36b365
-Winston
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
