Le mercredi 02 avril 2014 21:21:14, Dražen Odobašić a écrit : > On 02.04.2014 20:29, Even Rouault wrote: > > Le mardi 01 avril 2014 19:26:09, Dražen Odobašić a écrit : > >> Hi all, > >> > >> I've been successfully reading OSM PBF files using GDAL Python bindings, > >> even files larger than 5Gb. I did have to use > >> OGR_INTERLEAVED_READING=YES and OSM_USE_CUSTOM_INDEXING=NO, but > >> eventually OSM PBF is read and I can extract data. > > > > Generally, you shouldn't have to mess with OSM_USE_CUSTOM_INDEXING. > > > > As far as OGR_INTERLEAVED_READING=YES is concerned, it is not sufficient > > to define the environmenent variable. I invite you to read again > > "Interleaved reading" in http://gdal.org/ogr/drv_osm.html. The sample > > C++ code (ported to Python) must be used *exactly*. I really mean it. > > That's what ogr2ogr does with OSM data. > > Hi Even, > > this is the Python code that implements interleaved reading, which is more > or less copied from the c++ example in the docs: > > """ > # prepare for the layer/feature iteration > nLayerCount = ds.GetLayerCount() > thereIsDataInLayer = True > > while thereIsDataInLayer: > thereIsDataInLayer = False > > # read data from layers using OGR_INTERLEAVED_READING method > for iLayer in xrange(nLayerCount): > lyr = ds.GetLayer(iLayer) > > # read next Feature > feat = lyr.GetNextFeature() > while(feat is not None): > # continue reading features from this layer > thereIsDataInLayer = True > > # do something with the feature > # save_feature(feat) > > # force feature removal - required for INTERLEAVED_READING > feat.Destroy() > # get the next feature > feat = lyr.GetNextFeature() > """ >
That looks fine. > I had no issues when reading large PBF files using this method. For the > large files, without OSM_USE_CUSTOM_INDEXING=NO I got lots of "Cannot read > node ..." error messages, and I'm not sure why. Basically I stumbled onto > OSM_USE_CUSTOM_INDEXING=NO which seems to help. However, in this case > custom indexing has no effect, script still fails to extract data. Hum, there might be a bug in custom indexing that you've hit. What is the PBF file you are converting ? And to be sure to understand: when setting OSM_USE_CUSTOM_INDEXING=NO, the conversion works ? > > Is there anything else I could try ? > > Dražen -- Geospatial professional services http://even.rouault.free.fr/services.html _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
