Re: [R] Big Data reading subsample csv

2012-08-16 Thread Greg Snow
The read.csv.sql function in the sqldf package may make this approach quite simple. On Thu, Aug 16, 2012 at 10:12 AM, jim holtman wrote: > Why not put this into a database, and then you can easily extract the > records you want specifying the record numbers. You play the one time > expense of cr

Re: [R] Big Data reading subsample csv

2012-08-16 Thread jim holtman
Why not put this into a database, and then you can easily extract the records you want specifying the record numbers. You play the one time expense of creating the database, but then have much faster access to the data as you make subsequent runs. On Thu, Aug 16, 2012 at 9:44 AM, Tudor Medallion

[R] Big Data reading subsample csv

2012-08-16 Thread Tudor Medallion
Hello, I'm most grateful for your time to read this. I have a uber size 30GB file of 6 million records and 3000 (mostly categorical data) columns in csv format. I want to bootstrap subsamples for multinomial regression, but it's proving difficult even with my 64GB RAM in my machine and twice tha