Thx William and Brian for your swift responses, very insightful. I'll have
to hunt for more memory.
Cheers
Joris
On Tue, Sep 18, 2018 at 6:16 PM William Dunlap wrote:
> The ratio of object size to rds file size depends on the object. Some
> variation is due to how header information is stored
The ratio of object size to rds file size depends on the object. Some
variation is due to how header information is stored in memory and in the
file but I suspect most is due to how compression works (e.g., a vector of
repeated values can be compressed into a smaller file than a bunch of
random by
Your RDS file is likely compressed, and could have compression of 10x
or more depending on the composition of the data that is in it and the
compression method used. 'gzip' compression is used by default.
--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
On Tue,
Dear all,
I tried to read in a 3.8Gb RDS file on a computer with 16Gb available
memory. To my astonishment, the memory footprint of R rises quickly to over
13Gb and the attempt ends with an error that says "cannot allocate vector
of size 5.8Gb".
I would expect that 3 times the memory would be eno
