Re: [gdal-dev] Python-OGR not looping over more than 500 PGSQL features

2009-10-09 Thread Even Rouault
Selon Even Rouault : Hmm, of course, read lyr2 = ds.GetLayerByName('testhuge2') in the last snippet. > Selon Matthieu Rigal : > > Matthieu, > > I've been able to reproduce the behaviour you've observed by running the > following snippet : > > >>> > import ogr > > ds = ogr.Open('PG:dbname=autotest

Re: [gdal-dev] Python-OGR not looping over more than 500 PGSQL features

2009-10-09 Thread Even Rouault
Selon Matthieu Rigal : Matthieu, I've been able to reproduce the behaviour you've observed by running the following snippet : >>> import ogr ds = ogr.Open('PG:dbname=autotest') lyr = ds.GetLayerByName('testhuge') feat = lyr.GetNextFeature() count = 0 while feat is not None: count = count +

Re: [gdal-dev] Python-OGR not looping over more than 500 PGSQL features

2009-10-09 Thread Chaitanya kumar CH
Matthieu, I looked at the OGR's pg driver code. It is retrieving the data 500 records at a time using cursors. Can you provide some more lines from the log files, before and after the given two lines? In the mean time here is a quick and dirty solution. Recompile gdal after setting CURSOR_PAGE to

Re: [gdal-dev] Python-OGR not looping over more than 500 PGSQL features

2009-10-09 Thread Matthieu Rigal
Chaitanya, I looked into the log files and could see the following lines : 2009-10-09 15:23:56 CEST lua_db lua_admin STATEMENT:  FETCH 500 in OGRPGResultLayerReader 2009-10-09 15:23:56 CEST lua_db lua_admin STATEMENT:  CLOSE OGRPGResultLayerReader So it is OGR which is requesting only 500 featu

Re: [gdal-dev] Python-OGR not looping over more than 500 PGSQL features

2009-10-09 Thread Chaitanya kumar CH
Matthieu, GetFeatureCount in postgres driver is implemented with the COUNT() function. ( http://trac.osgeo.org/gdal/browser/trunk/gdal/ogr/ogrsf_frmts/pg/ogrpgtablelayer.cpp#L1895 ) So it was never returning more than 500 features. IMHO, the number 500 looks like an intentional limitation, probab

[gdal-dev] Python-OGR not looping over more than 500 PGSQL features

2009-10-09 Thread Matthieu Rigal
Hi people, I recently discovered a strange behaviour which was not happening before, I think... I'm using Python 2.5 with GDAL/OGR 1.6 and PGsql 8.3 on Linux. I am reading a table containing 858 features with several columns and geometry. I first do a SELECT on the table and the GetFeatureCount