Re: [R] Parsing large amounts of csv data with limited RAM

2015-07-14 Thread jim holtman
take a look at the sqldf package because it has the ability to load a csv file to a database from which you can then process the data in pieces Jim Holtman Data Munger Guru What is the problem that you are trying to solve? Tell me what you want to do, not how you want to do it. On Tue, Jul 14,

Re: [R] Parsing large amounts of csv data with limited RAM

2015-07-14 Thread Jeff Newmiller
You seem to want your cake and eat it too. Not unexpected, but you may have your work cut out to learn about the price of having it all. Plotting: pretty silly to stick with gigabytes of data in your plots. Some kind of aggregation seems required here, with the raw data being a stepping stone to

[R] Parsing large amounts of csv data with limited RAM

2015-07-14 Thread Dupuis, Robert
I'm relatively new to using R, and I am trying to find a decent solution for my current dilemma. Right now, I am currently trying to parse second data from a 7 months of CSV data. This is over 10GB of data, and I've run into some "memory issues" loading them all into a single dataset to be plot