Hi,
please consider the following minimal reproducible example:
Create a new R package which just contains the following two (exported) objects:
crash_dumps <- new.env()
f <- function() {
x <- runif(1e5)
dump <- lapply(1:2, function(i) unserialize(serialize(sys.frames(), NULL)))
assign("last.dump", dump, crash_dumps)
}
WARNING: the following will probably eat all your RAM!
Attach this package and run:
for (i in 1:100) {
print(i)
f()
}
You will notice that with each iteration the execution of f() slows down
significantly while the memory consumption of the R process (v4.0.5 on Linux)
quickly explodes.
I am having a hard time to understand what exactly is happening here. Something
w.r.t. too deeply nested environments? Could someone please enlighten me?
Thanks!
Regards,
Andreas
Background:
In an R package I store crash dumps on error in a parallel processes in a way
similar to what I have just shown (hence the (un)serialize(), which happens as
part of returning the objects to the parent process). The first 2 or 3 times I
do so in a session everything is fine, but afterwards it takes very long and I
soon run out of memory.
Some more observations:
- If I omit `x <- runif(1e5)`, the issues seem to be less pronounced.
- If I assign to .GlobalEnv instead of crash_dumps, there seems to be no issue
- probably because .GlobalEnv is not included in sys.frames(), while
crash_dumps is indirectly via the namespace of the package being the parent.env
of some of the sys.frames()!?
- If I omit the lapply(...), i.e. use `dump <-
unserialize(serialize(sys.frames(), NULL))` directly, there seems to be no
issue. The immediate consequence is that there are less sys.frames and - in
particular - there is no frame which has the base namespace as its parent.env.
- If I make crash_dumps a list and use assignInMyNamespace() to store the dump
in it, there also seems to be no issue. I will probably use this as a
workaround:
crash_dumps <- list()
f <- function() {
x <- runif(1e5)
dump <- lapply(1:2, function(i) unserialize(serialize(sys.frames(), NULL)))
crash_dumps[["last.dump"]] <- dump
assignInMyNamespace("crash_dumps", crash_dumps)
}
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
