Jim At the moment I'm using write.table. I tried using write.matrix from the MASS package, but that failed. Integers are not appropriate here because we are working with fractions of miles for some locations and that needs to be retained. The range is from 0 to about 3.5 (it's a little less than that with the digits)
I haven't tried the save function yet, but I wasn't aware of that one previously. Thanks for pointing that out. The bigger concern is reading and working with that dataset in the other calculation though. Adrienne On Thu, Jun 5, 2014 at 9:37 AM, jim holtman <jholt...@gmail.com> wrote: > How are you writing it out now? Are you using 'save' which will compress > the file? What are the range of numbers in the matrix? Can you scale them > to integers (what is the range of the numbers) which might save some space? > You did not provide enough information to make a definitive solution. > > > Jim Holtman > Data Munger Guru > > What is the problem that you are trying to solve? > Tell me what you want to do, not how you want to do it. > > > On Thu, Jun 5, 2014 at 9:26 AM, Adrienne Wootten <amwoo...@ncsu.edu> > wrote: > >> All, >> >> Got a tricky situation and unfortunately because it's a big file I can't >> exactly provide an example, so I'll describe this as best I can for >> everyone. >> >> I have a distance matrix that we are using for a modeling calculation in >> space for multiple days. Since the matrix is never going to change for >> different dates, I want to keep the matrix in a file and refer to that so >> I >> don't have to repeat the calculation over and over again for that. The >> problem is it's a 32000 X 32000 matrix and roughly works out to 15GB of >> storage. This makes it a trick to read the file back into R, but it >> leaves >> me with two questions for the group. >> >> Is there anyway to have R write this out so that it takes up less space? >> I >> know R primarily treats numbers as doubles, but I'm trying to find a way >> to >> get R to write the values as floats or singles. >> >> with how big it is, it may not be wise to save it as an object in R when >> read in, so I'm wondering is there anyway to have R do the calculation it >> needs to do without saving the matrix as an object in R? Basically can I >> have it run the calculation off the file itself? >> >> Thanks! >> >> Adrienne >> >> >> -- >> Adrienne Wootten >> Graduate Research Assistant >> State Climate Office of North Carolina >> Department of Marine, Earth and Atmospheric Sciences >> North Carolina State University >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > -- Adrienne Wootten Graduate Research Assistant State Climate Office of North Carolina Department of Marine, Earth and Atmospheric Sciences North Carolina State University [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
