Re: [R] Need help: Loading a large data set.

2008-11-23 Thread roger koenker
Both SparseM and Matrix have facilities for rbind and cbind that allow you to concatenate pieces of sparse matrices together. On Nov 23, 2008, at 2:19 PM, Atul Kulkarni wrote: Hi All, I am dealing with a large data set which translates in to a sparse matrix, I want to load this data that i

Re: [R] Need help: Loading a large data set.

2008-11-23 Thread jim holtman
What matrix package are you using? I have not used sparse matrices, but a quick look at the help file of Matrix talks about a file format for reading in a sparse matrix. I would assume that all you need to do is to read in your files and write them out in that format. You can do it in using 'lis

[R] Need help: Loading a large data set.

2008-11-23 Thread Atul Kulkarni
Hi All, I am dealing with a large data set which translates in to a sparse matrix, I want to load this data that is spread in approximately 17000+ files each defining a row and each file has variable number of records that come with its column number and the value that they store. I wish to load