Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Even Rouault
Le mercredi 03 août 2011 20:28:35, Jay L. a écrit : > Evan, > > To ensure that i understand here is an example: > > If I have a GTiff, where the block size is one row by all of the columns > (a single scanline), I should try to read in either one scanline at a > time, or multiple entire scanline

Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Jay L.
Evan, To ensure that i understand here is an example: If I have a GTiff, where the block size is one row by all of the columns (a single scanline), I should try to read in either one scanline at a time, or multiple entire scanlines. It is inefficient to take say 10 rows and only half of the col

Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Even Rouault
Le mercredi 03 août 2011 17:32:53, Antonio Valentino a écrit : > Hi Jay, > > Il 03/08/2011 16:53, Jay L. ha scritto: > > I have been working on this problem as well. Initially, the attempt was > > to ReadAsArray small chunks. Unfortunately this is quite inefficient. > > Someone more knowledgeab

Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Alexander Bruy
Hi 2011/8/3 Antonio Valentino : > In my experience using too large chunks of memory can cause, > paradoxically, slowdowns. > My suggestion is to define a reasonable maximum size for arrays in your > application/library and switch to the "fallback algorithm for large > rasters" every time that MAX_

Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Even Rouault
Le mercredi 03 août 2011 15:35:22, Alexander Bruy a écrit : > Hi, > > There is a well-know "problem": reading really large rasters or bands > into memory with DataSource.ReadAsArray() method impossible due > memory limitations. For example, when I try to read one band with > size 53109x29049 I get

Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Antonio Valentino
Hi Jay, Il 03/08/2011 16:53, Jay L. ha scritto: > I have been working on this problem as well. Initially, the attempt was to > ReadAsArray small chunks. Unfortunately this is quite inefficient. Someone > more knowledgeable will know why, but I suspect it has to do with either > thrashing or the

Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Jay L.
I have been working on this problem as well. Initially, the attempt was to ReadAsArray small chunks. Unfortunately this is quite inefficient. Someone more knowledgeable will know why, but I suspect it has to do with either thrashing or the fact that full blocks are not being read in (as is the c

Re: [gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Antonio Valentino
Hi Alexander, Il 03/08/2011 15:35, Alexander Bruy ha scritto: > Hi, > > There is a well-know "problem": reading really large rasters or bands > into memory with DataSource.ReadAsArray() method impossible due > memory limitations. For example, when I try to read one band with > size 53109x29049 I

[gdal-dev] Raster size and ReadAsArray()

2011-08-03 Thread Alexander Bruy
Hi, There is a well-know "problem": reading really large rasters or bands into memory with DataSource.ReadAsArray() method impossible due memory limitations. For example, when I try to read one band with size 53109x29049 I get error: File "C:\OSGeo4W\apps\Python25\lib\site-packages\osgeo\gdal.py"