Thanks Brent, your suggestion worked.
On Jul 29, 6:23 pm, Brent Pedersen <[email protected]> wrote: > I would guess your first example doesn't show what you expect > because you truncated the file. > > you can use the glob module to get all .gz files and iterate trough: > something like below (adjusting for your windows paths). I think you have to > .read() the entire file into memory and use numpy's fromstring() since > it doesn't > recognize the gzip file handle in fromfile. > > import gzip > import glob > import numpy as np > > num_lon = 10 > num_lat = 10 > > def read_bin(gz, num_lon, num_lat): > arr = np.fromstring(gz.read(), dtype='f').reshape(num_lon, num_lat) > return arr > > for gz in (gzip.open(f) for f in glob.glob("/tmp/*.gz")): > arr = read_bin(gz, num_lon, num_lat) > print arr[5, 3] > > On Fri, Jul 29, 2011 at 4:45 AM, Hanlie Pretorius > > <[email protected]> wrote: > > Hi, > > > I'm working on Windows XP with Python 2.6. > > > I need to read and process hundreds of GSMaP binary files that are in the > > .gz archive format. > > > I found a site (http://www.doughellmann.com/PyMOTW/gzip/) and tried > > their code with two files: one of the hundreds of files that I need to > > process (f1 below) and one that I created with 7-Zip from a text file > > that contains the text 'Text to test gzip module.' (f2 below). The > > code and the output follow: > > > [code] > > import gzip > > > f1 = 'GSMaP_MVK+.20050101.00.0.1deg.hourly.v484.gz' > > f2 = ''text.txt.gz' > > if1 = gzip.open(f1, 'rb') > > if2 = gzip.open(f2,'rb') > > try: > > print if1.read() > > print 'done with f1' > > print if2.read() > > print 'done with f2' > > finally: > > if1.close() > > if2.close() > > [/code] > > > [output] > > done with f1 > > Text to test gzip module. > > done with f2 > > [/output] > > > This seems to indicate that something is wrong with f1 (the GSMaP > > file), but I can unzip the file manually and read it with a python > > script (code and output pasted after signature). I have hundreds of > > GSMAP files that have unique archived file names, but they all unzip > > to the same binary file, so I have to process the archived files in > > the python script. > > > I would be grateful if someone could help me achieve this. > > > Regards > > Hanlie > > > [code] > > import numpy as np > > import array > > import gzip > > > # define longitude and latitude grid resolution > > num_lon = 3600 > > num_lat = 1200 > > > # read binary file and store its data in an array > > fp = 'C:\\out.00' > > tmpfile = open(fp, 'rb') > > binvalues = array.array('f') > > binvalues.read(tmpfile, num_lon * num_lat) > > data = np.array(binvalues) > > data = np.reshape(data, (num_lat, num_lon)) > > > tmpfile.close() > > > # read one value from array > > rows=[584] > > cols=[30] > > print data[rows,cols] > > [/code] > > > [output] > > 0.12709916 > > [/output]
