Re: [R] read multiple large files into one dataframe

2010-01-25 Thread Brad Patrick Schneid
Thats it Hadley!!! Thank you. -- View this message in context: http://n4.nabble.com/read-multiple-large-files-into-one-dataframe-tp891835p1290089.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https:

Re: [R] read multiple large files into one dataframe

2010-01-25 Thread hadley wickham
On Mon, Jan 25, 2010 at 4:43 AM, Paul Hiemstra wrote: > Brad Patrick Schneid wrote: >> >> ###  The following is very helpful # listOfFiles <- >> list.files(pattern= ".txt") d <- do.call(rbind, lapply(listOfFiles, >> read.table)) ### >> >> but what if each file c

Re: [R] read multiple large files into one dataframe

2010-01-25 Thread Paul Hiemstra
Brad Patrick Schneid wrote: ### The following is very helpful # listOfFiles <- list.files(pattern= ".txt") d <- do.call(rbind, lapply(listOfFiles, read.table)) ### but what if each file contains information corresponding to a different subject and I need t

Re: [R] read multiple large files into one dataframe

2010-01-24 Thread Brad Patrick Schneid
### The following is very helpful # listOfFiles <- list.files(pattern= ".txt") d <- do.call(rbind, lapply(listOfFiles, read.table)) ### but what if each file contains information corresponding to a different subject and I need to be able to tell where each

Re: [R] read multiple large files into one dataframe

2009-05-13 Thread Liaw, Andy
A few points to consider: - If all the data are numeric, then use matrices instead of data frames. - With either data frames or matrices, there is no way (that I'm aware of anyway) in R to stack them without making at least one copy in memory. - Since none of the files has a header row, I would

Re: [R] read multiple large files into one dataframe

2009-05-13 Thread Simon Pickett
can you provide reproducible code please? even a fake example would help. I would 1) set up a loop to read in each file from a directory 2) inside the loop chop up/ aggregate the data, each file in turn and spit each new aggreagated file out to a directory using write.table(). This will redu

Re: [R] read multiple large files into one dataframe

2009-05-13 Thread Mike Lawrence
What types of data are in each file? All numbers, or a mix of numbers and characters? Any missing data or special NA values? On Wed, May 13, 2009 at 7:45 AM, SYKES, Jennifer wrote: > Hello > > > > Apologies if this is a simple question, I have searched the help and > have not managed to work out

Re: [R] read multiple large files into one dataframe

2009-05-13 Thread baptiste auguie
I'd first try plyr and see if it's efficient enough, library(plyr) listOfFiles <- list.files(pattern= ".txt") d <- ldply(listOfFiles, read.table) str(d) alternatively, d <- do.call(rbind, lapply(listOfFiles, read.table)) HTH, baptiste On 13 May 2009, at 12:45, SYKES, Jennifer wrote: