Steve Powers <powers_s <at> nd.edu> writes: > > Hello, > > This one has been bugging me for a long time and I have never found a > solution. I am using R version 2.15.1 but it has come > up in older versions of R I have used over the past 2-3 years. > > Q: Am I wrong to expect that R should handle hundreds of iterations of the > base model or statistical functions, embedded within for loops, in one > script run? I have found that when I write scripts that do this, sometimes > they have a tendency to crash, seemingly unpredictably. > > For example, one problem script of mine employs glm and gls about a hundred > different times, and output files are being written at the end of each > iteration. I have used my output files to determine that the crash cause is > not consistent (R never fails at the same iteration). Note that the data are > fixed here (no data generation or randomization steps, so that is not the > issue). But it is clear that scripts with larger numbers of iterations are > more likely to produce a crash. > > And a year or two ago, I had a seemingly stable R script again with for > looped model fits, but discovered this script was prone to crashing when I > ran it on a newer PC. Because the new PC also seemed to be blazing through R > code absurdly fast, I tried adding a short "fluff" procedure at the end of > each iteration that required a few seconds of processing time. Low and > behold, when I added that, the script stopped crashing (and each iteration > of course took longer). I still don't understand why that fixed things.
All of the advice given so far is useful (I think), but I thought I would chime in and answer one of your questions, which is that this is indeed surprising, especially if you're using only base packages. I and many other people routinely run thousands of iterations of these types of analyses with no problem. If (as Brian Ripley suggested) "crash" just means that some of your scripts stop with errors in some cases, then that's *not* surprising -- there are lots of ways to get glm() and gls() to give errors with slightly weird data sets. However, having R actually crash (i.e. the whole R session 'terminates abnormally', in the words of the posting guide) is quite a bit more unusual, and (if you are only using base R, not any contributed packages that may call compiled code in bad ways [or calling your own compiled code]) *always* constitutes a bug. Nondeterministic behavior in a deterministic function (i.e. no random number generation) is also unusual/surprising. The key here is finding a reproducible example, which can be tough for these kinds of problems. But it sounds like you have one, so if you can trim it down to a manageable size (see http://tinyurl.com/reproducible-000 for tips on creating reproducible examples), and give full information about your system (as suggested by another poster), this would be of great interest, especially to the people who hang out on r-de...@r-project.org . My prior probabilities are fairly strongly on the problem being something flaky about your system, but helping to identify these kinds of bugs in R is a community service ... ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
