Duncan,
Thank you very much for your help. Your first work-around solved my
problem, even if the second one didn't. I'm not sure if it was the
elimination of the file() or the close() commands, or both, that did
it.
You got me going again after a couple of weeks of casting about. My hat
is off to you!
Thanks,
Rich
----- Original Message -----
From: "Duncan Murdoch" <murd...@stats.uwo.ca>
To: "Marilyn & Rich Short" <rm.sh...@comcast.net>
Cc: <r-help@r-project.org>
Sent: Saturday, July 18, 2009 8:21 AM
Subject: Re: [R] Problem With Repeated Use Of Load/Save/Close Commands
On 18/07/2009 11:08 AM, Duncan Murdoch wrote:
On 17/07/2009 7:57 PM, Marilyn & Rich Short wrote:
Hello,
I'm having a problem in R with repeated use of the "load", "save",
and "close" commands. I'm getting an error message that reads, "Too
many open files". I'm opening files and closing them (and unlinking
them), but when I go through that process 509 times, the program
halts and I get this error message: "cannot open the connection"
with warning messages: "Too many open files". I've been working on
this problem for a couple of weeks and have gleaned a bit of info
from different internet threads, but no solutions yet.
I'm using Windows XP, SP3 and R 2.9.1.
Here is my session info:
R version 2.9.1 (2009-06-26)
i286-pc-mingw32
locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States. 1252;LC_MONETARY=English_United
States. 1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
I'm using the vanilla load of R, with nothing added after booting
up.
The problem also occurs on my Vista machine as well.
The program below will induce the problem.
------------------------------------------------------------
junk = 1
PathA = tempdir()
conX = paste(PathA,"junk",sep="\\")
outJunk = file(conX, open="wb")
save(junk, file=outJunk)
close(outJunk)
for(i in 1:4000){
outMIA = file(conX, open="rb")
load(file=outMIA)
close(outMIA)
closeAllConnections()
unlink(conX)
rm(outMIA)
rm(junk)
cat(" i = ",i,sep=" ")
gc()
zzz = showConnections(all=FALSE)
cat(" zzz = ",zzz,"\n",sep=" ")
}
--------------------------------------------------------------------
There is some talk on the internet that some windows systems have a
limit of 512 files that can be open at one time. Even though I'm
closing my files each time, something is keeping track of how many
times I've opened and closed a file in a session. I've talked to
Microsoft and run a test program in Visual Studio C#, and, at the
moment, it looks like the problem does not lie in the Microsoft
arena. The C# program performed a similar task 10,000 times without
a problem. I'm not totally convinced, but the current evidence says
to look elsewhere.
I've attached a script that will induce the problem. However, be
warned that, if you use it, you will have to get out of R after you
run it. R will no longer be able to access files such as help or
sessionInfo(). This can be solved by getting out of the R GUI and
back in.
R E-mails from as far back as 2006 ask for help on the issue. There
have been several suggestions, but no confirmed solution that I can
find. You will see my attempt at these suggestions in the script [
rm(outMIA); rm(junk); closeAllConnections(); and gc(); after
close(outMIA); and unlink(conX);]. For me, this becomes important
because it limits the total number if iterations in nested do-loops.
This is where I ran into the problem. The program above will allow
you to reproduce the problem.
Any suggestion would be greatly appreciated.
I've somewhat localized the problem. load(file=outMIA) passes the
outMIA connection to gzcon(), which handles decompression of the
data. gzcon() re-opens the file, and it looks as though the original
file handle is lost, because closing either the result of gzcon() or
the original connection results in only the second file handle being
closed.
So a second workaround besides the one I sent earlier is just not to
open or close outMIA. That is, rewrite it as
junk = 1
PathA = tempdir()
conX = paste(PathA,"junk",sep="\\")
outJunk = file(conX, open="wb")
save(junk, file=outJunk)
close(outJunk)
for(i in 1:4000){
outMIA = file(conX)
load(file=outMIA)
rm(outMIA)
rm(junk)
cat(" i = ",i,sep=" ")
gc()
zzz = showConnections(all=FALSE)
cat(" zzz = ",zzz,"\n",sep=" ")
}
so that load() does the open and close, and things are fine.
Oops, not quite fine. There's a whole set of warnings...
Duncan Murdoch
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
