The R 'save' format (as used for the saved workspace .RData) is
described in the 'R Internals' manual (section 1.8). It is intended
for R objects, and you would first have to create one[*] of those in
your other application. That seems a lot of work.
The normal way to transfer numeric data between applications is to
write a binary file: R can read such files with readBin(), and it also
has wrappers/C-code to read a number of commmon binary data formats
(e.g. those from SPSS).
With character data there are more issues (and more formats, see also
readChar()), but load() is not particularly fast for those.
Ultimately the R functions pay a performance price for their
flexibility so hand-crafted C code to read the format can be
worthwhile: but see the comments below about whether I/O speed is
that important.
[*] the 'save' format is a serialization of a single R object, even if
you save many objects, since the object(s) are combined into a
pairlist.
On Sun, 8 Nov 2009, Adam D. I. Kramer wrote:
Hello,
I frequently have to export a large quantity of data from some
source (for example, a database, or a hand-written perl script) and then
read it into R. This occasionally takes a lot of time; I'm usually using
read.table("filename",comment.char="",quote="") to read the data once it is
written to disk.
Specifying colClasses and nrows will usually help.
To read from a database, packages such as RODBC use binary data
transfer: with suitable tuning this can be fast.
However, I *know* that the program that generates the data is more
or less just calling printf in a for loop to create the csv or tab-delimited
file, writing, then having R parse it, which is pretty inefficient. Instead,
I am interested in figuring out how to write the data in .RData
format so that I can load() it instead of read.table() it.
Without more details it is hard to say if it is inefficient.
read.table() can read data pretty fast (millions of items per second)
if used following the hints in the 'R Data' manual. See e.g.
https://stat.ethz.ch/pipermail/r-devel/2004-December/031733.html
Almost anything non-trivial one might do with such data is much
slower. The trend is to write richer (and slower to read) data
formats.
Trolling the internet, however, has not suggested anything about the
specification for an .RData file. Could somebody link me to a specification
or some information that would instruct me on how to construct a .RData
file (either compressed or uncompressed)?
Also, I am open to other suggestions of how to get load()-like
efficiency in some other way.
Many thanks,
Adam D. I. Kramer
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
