This seriously looks like a bug in load(): the gzcon() magic in the code
leaves the connection open. On Linux x86_64 I can do:
file.name <- "/tmp/foo.RData"
data <- runif(10)
save(data, file=file.name)
for ( i in 1:4000 ) { con <- gzfile(file.name, "rb"); load(file=con);
close(con); }
## No problem!
for ( i in 1:4000 ) { con <- file(file.name, "rb"); load(file=con);
close(con); }
# Error in load(file = con) : cannot open the connection
# In addition: Warning messages:
# 1: In gzcon(file) : cannot open file '/tmp/foo.RData': Too many open files
# 2: In load(file = con) :
# cannot open file '/tmp/foo.RData': Too many open files
print(i)
# [1] 1021
showConnections()
# description class mode text isopen can read can write
M&R: Use the gzfile() workaround above.
R-helpers: (a) is this a bug? (b) where does one report bugs in core R
anyhow?
Allan
On 16/07/09 21:55, Marilyn & Rich Short wrote:
Hello,
I'm having a problem in R. I'm getting an error message that reads,
"Too many open files". I'm opening files and closing them (and
unlinking them), but when I go through that process 509 times, the
program halts and I get this error message: "cannot open the
connection" with warning messages: "Too many open files". I've been
working on this problem for a couple of weeks and have gleaned a bit
of info from different internet threads, but no solutions yet.
I'm using Windows XP, SP3 and R 2.9.1.
Here is my session info:
sessionInfo()
R version 2.9.1 (2009-06-26)
i386-pc-mingw32
locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States.1252;LC_MONETARY=English_United
States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
(The problem also occurs on my Vista machine as well.)
There is some talk on the internet that some windows systems have a
limit of 512 files that can be open at one time. Even though I'm
closing my files each time, something is keeping track of how many
times I've opened and closed a file in a session. I've talked to
Microsoft and run a test program in Visual Studio C#, and, at the
moment, it looks like the problem does not lie in the Microsoft arena.
The C# program performed a similar task 10,000 times without a
problem. I'm not totally convinced, but the current evidence says to
look elsewhere.
I've attached a script that will induce the problem. However, be
warned that, if you use it, you will have to get out of R after you
run it. R will no longer be able to access files such as help or
sessionInfo(). This can be solved by getting out of the R GUI and back
in. Also, I'm using my pathname, "MyPathnameA", so you'll probably
want to edit it with your own.
R E-mails from as far back as 2006 ask for help on the issue. There
have been several suggestions, but no confirmed solution that I can
find. You will see my attempt at these suggestions in the script [
rm(outMIA);rm(junk); and gc(); after close(outMIA); and
unlink(FileNameMIA);]. For me, this becomes important because it
limits the total number if iterations in nested do-loops. This is
where I ran into the problem. The program below is a distillation from
the original program.
Any suggestion would be greatly appreciated.
I'm a retired engineer and am picking up R to use on a genetic
algorithm I want to play with.
Thanks for your attention,
Rich Short
# This script induces an error: "cannot open the connection" with
# warnings: "Too many open files".
?
# ---------------- Create a junk file and "save" it for use further
into the program ---
junk <- 1
MyPathnameA <- "C:\\Documents and Settings\\All
Users\\Documents\\SIREPO\\DBFS-0150"
connectionX <- paste(MyPathnameA,"junk",sep="\\")
outJunk <- file(connectionX, open="wb")
save(junk, file=outJunk)
close(outJunk)
# The next two lines are a repeat from above. They will be useful if
you want to run
# this script again after "junk" has been loaded.
MyPathnameA <- "C:\\Documents and Settings\\All
Users\\Documents\\SIREPO\\DBFS-0150"
connectionX <- paste(MyPathnameA,"junk",sep="\\")
FileNameMIA <- connectionX
for(i in 1:4000){
# ----------------- Load "junk" ---------------------------------
outMIA <- file(FileNameMIA, open="rb")
load(file=outMIA) # load a file
# ----------------- close "junk"; unlink; remove; ----------------
close(outMIA) # close the file
unlink(FileNameMIA) # Unlink
rm(outMIA) # This should be unnecessary. I
tried it just to be sure the retention of the variable was
rm(junk) # not causing the connection
to be kept open.
cat(" i = ",i,sep=" ") # Show what iteration we are on.
gc() # Garbage collection. This
should be unnecessary. Another failed attempt at a work-around.
zzz <- showConnections(all=FALSE)
cat(" zzz = ",zzz,"\n",sep=" ") # This screen print shows that
there are no open files
}
?
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
