Albert-Jan Roskam wrote:
Hi,
I very recently started using R (as in: last week) and I was wondering if anyone could point me to website(s) with sample code to deal with large datasets (length- and/or breadthwise). I understood that R was never designed to work with datasets larger than, say, a couple of hundred Mb. One way is (as I also read) to let R work in conjunction with SQL. That's one interesting approach I'd like to know more about. But I was also hoping that there also were pure R solutions for working with very large tables (was 'scan' designed for that?). In any case, a standard approach would be desirable.
Hi Albert-Jan,
If you are faced with enormous datasets, R is, in my opinion, a great
tool. It only takes careful thought how to tackle an analysis if the
data does not fit into your memory all at once. As you mentioned, you
could put your data in a database and extract subsets to do the
analysis, lateron combining the results. You could sparsely sample your
data. But without you specifying exactly what it is you want to do, it
is impossible for us to give you any specific advice. Please review the
posting guide for some hints as to what kind of information you can
provide us with.
cheers,
Paul
Thanks in advance.
Cheers!!
Albert-Jan
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In the face of ambiguity, refuse the temptation to guess.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[[alternative HTML version deleted]]
------------------------------------------------------------------------
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Drs. Paul Hiemstra
Department of Physical Geography
Faculty of Geosciences
University of Utrecht
Heidelberglaan 2
P.O. Box 80.115
3508 TC Utrecht
Phone: +3130 274 3113 Mon-Tue
Phone: +3130 253 5773 Wed-Fri
http://intamap.geo.uu.nl/~paul
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
