Hi,
I first posted this message on r-help, but got redirected here.
I encounter a strange memory error, and I'd like some help to determine if I'm
doing something wrong or if there's a bug in recent R versions...
I'm currently working on a DeepNet project @home, with an old PC with 4Gb RAM,
running Ubuntu 16.04.
For efficiency reason, I preprocessed my dataset and stored it as a csv file
with write.csv() so that I can reload it at will with read.csv(). I did it
several time, everything was working fine.
A few days ago, I tried to pursue my work on anther machine @work, I wanted to
use a more recent & powerful machine with 8Gb of RAM running under Ubuntu
16.10, but I ran into a strange error:
$ R
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)
Error: C stack usage 7970548 is too close to the limit
I read a few fora on the Internet and found a potential workaround, consisting
in increasing the stack size using ulimit. Unfortunately, it doesn't work for
me:
$ ulimit -s
8192
$ ulimit -s $((100*$(ulimit -s)))
$ R --vanilla
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)
This was under Ubuntu 16.10 with R version 3.3.1 (2016-06-21) "Bug in Your Hair"
Yesterday, I upgraded my Ubuntu to 17.04 (R version 3.3.2 (2016-10-31) "Sincere
Pumpkin Patch") and tried again. This resulted in the exact same error.
How is it possible that a 513MB file cannot be read on a machine with 8GB RAM?
Also, how is it possible that a machine with twice the RAM as the previous one
cannot load the same file?
Since the only other difference is the Ubuntu version (thus R version), I
assume there's a bug in R/csv loader, but I don't know where to look for...
If anyone has an idea I would be glad to hear it...
GM
--------
For the sake of completeness, I share my Trainset at the following link:
> https://mega.nz/#!ZMs0TSRJ!47DCZCnE6_FnICUp8MVS2R9eY_GdVIyGZ5O9TiejHfc
FYI, it loads perfectly on 2 machines with Ubuntu 16.04 and R version 3.2.3
(2015-12-10) -- "Wooden Christmas-Tree". But exceededs stack mem under Ubuntu
16.10's R version 3.3.1 (2016-06-21) -- "Bug in Your Hair" and Ubuntu to
17.04's R version 3.3.2 (2016-10-31) "Sincere Pumpkin Patch".
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
