Thanks a lot, Prof. Ripley. the problem must be download.file() prior R-patched 2.12. for each loop, I tired a couple of candidate links and only one or none will work, this must accumulate soon to an amount of unclosed destination files beyond the tolerance of windows. I updated R from 2.10 to R-patched 2.12 and the problem gone.
On Thu, Nov 25, 2010 at 3:09 AM, Prof Brian Ripley <rip...@stats.ox.ac.uk> wrote: > We don't have any of the information asked for in the posting guide, such as > your version of R, reproducible example .... > > But please try R-patched, since this might be > > • download.file() could leave the destination file open if the URL > was not able to be opened. (PR#14414) > > (If you had followed the posting guide you would have tried R-patched before > posting ....) > > > On Wed, 24 Nov 2010, Yong Wang wrote: > >> Dear List >> >> I asked this question before, got some tips but can't get it solved. > > Where? You didn't give a reference, and it would have helped the helpers. > >> it is clear now that this problem only occurs when run on windows (I >> tested it on windows XP) nothing wrong at all when run on unix. >> unfortunately, sometimes I have to run it on windows, >> I appreciate any suggestion on how to circumvent this problem when run >> on windows. >> below is the problem description I submitted before. >> >> ############################################################# >> >> I am running a loop downloading web pages and save the html to a >> temporary file (use download.file() ) >> then read (using readLines) it in for processing; >> finally write useful info from each processed page to a unique file >> >> the problem is once the loop runs up to somewhere near 5000, it will >> throw out an err like below and won't go further. >> >> ---------------------------------------------------------------- >> Error in file(file, ifelse(append, "a", "w")) : >> cannot open the connection >> ----------------------------------------------------------------- >> >> In the meantime, a request for new connection won't be successful, for >> example, a request for the help page of "file" will trigger err below >> >> ----------------------------------------------------------------------- >> ?file >> Error in gzfile(file, "rb") : cannot open the connection >> In addition: Warning message: >> In gzfile(file, "rb") : >> cannot open compressed file >> 'C:/PROGRA~1/R/R-211~1.1/library/stats/help/aliases.rds', probable >> reason 'Too many open files' >> ----------------------------------------------------------------------- >> >> I am not sure if the problem is too many connections not closed. since >> I close the file connection after each readLines. >> checking with showConnections(all=T) does not show excessive >> connections and closeAllConnections() does not help. >> >> Can any one help me on this? >> >> >> Any answer highly appreciated. >> >> yong >> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > -- > Brian D. Ripley, rip...@stats.ox.ac.uk > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > University of Oxford, Tel: +44 1865 272861 (self) > 1 South Parks Road, +44 1865 272866 (PA) > Oxford OX1 3TG, UK Fax: +44 1865 272595 ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
