Hi ,
thanks, the lower.tri idea is I guess the best way. Will try that.
Cheers
Martin
On 8/25/2010 2:21 PM, Sarah Goslee wrote:
Hi,
On Wed, Aug 25, 2010 at 8:11 AM, Martin Tomko<martin.to...@geo.uzh.ch> wrote:
Hi Sarah,
thank you very much for your answer. I have been spitting things out on
screen, but unfortunately I have not run it as a batch log, but in the
interactive window, so when the GUI crashed I was left without trace... I
guess I should google how to run it from a batch.
I have no idea how to do that in Windows, but I'm sure it's possible. :)
That way the things written to the screen will be saved in a file instead.
I should also explore using RData, I have ony been using csv files and
tables so far, it seems that can also bring some added performance.
If you need to save *everything*, RData is what you get when you use save(),
or when you close a session and choose y to saving the data. It can be
read in using load(). That's one way to be able to pick up where you left
off.
As the main output of my process is a matrix, I would really need to append
to a matrix after each iteration. I have identified the write.table append
parameter-based solution, but that would only append rows. Is there a way to
slowly "grow" a matrix in both directions, meaning append columns as well
(it is a big distance matrix).
AFAIK, you can't append columns that way because of the way text files are
written to disk. You'd need to rewrite the whole thing, or possibly write it out
in lower triangular format with NA values as padding (assuming it's a symmetric
distance).
Or for that matter, you could just write it out as a really long
vector, and turn
it back into a matrix later if you need to read the saved file in after a crash.
I'd recommend saving whatever variables are needed so that you can pick up
exactly where you left off, if possible. Much nicer to pick up 12 hours in than
to start over from the beginning.
Not R, but I just finished a 5-week batch job. You can bet that I put a lot of
thought into incremental save points and how to resume after an unexpected
halt!
Sarah
--
Martin Tomko
Postdoctoral Research Assistant
Geographic Information Systems Division
Department of Geography
University of Zurich - Irchel
Winterthurerstr. 190
CH-8057 Zurich, Switzerland
email: martin.to...@geo.uzh.ch
site: http://www.geo.uzh.ch/~mtomko
mob: +41-788 629 558
tel: +41-44-6355256
fax: +41-44-6356848
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
