Le mardi 01 avril 2014 19:26:09, Dražen Odobašić a écrit : > Hi all, > > I've been successfully reading OSM PBF files using GDAL Python bindings, > even files larger than 5Gb. I did have to use OGR_INTERLEAVED_READING=YES > and OSM_USE_CUSTOM_INDEXING=NO, but eventually OSM PBF is read and I can > extract data.
Generally, you shouldn't have to mess with OSM_USE_CUSTOM_INDEXING. As far as OGR_INTERLEAVED_READING=YES is concerned, it is not sufficient to define the environmenent variable. I invite you to read again "Interleaved reading" in http://gdal.org/ogr/drv_osm.html. The sample C++ code (ported to Python) must be used *exactly*. I really mean it. That's what ogr2ogr does with OSM data. > > If I use osmconvert [0] and osmfilter [1] to preprocess data, Python script > complains that OSM_USE_CUSTOM_INDEXING is not used. In the debug output, it > will report that OSM_USE_CUSTOM_INDEXING is used... > > """ > Tue Apr 1 19:04:56 2014: Shape: DBF Codepage = UTF-8 for > /tmp/out/world1.shp Tue Apr 1 19:04:56 2014: Shape: Treating as encoding > 'UTF-8'. > Tue Apr 1 19:04:57 2014: OSM: Using SQLite indexing for points > Tue Apr 1 19:04:57 2014: OGR: > OGROpen(/go_video/admin-levels.pbf/0x1d4a570) succeeded as OSM. > Tue Apr 1 19:04:57 2014: CPLError: 'admin_level' not recognised as an > available field. > ERROR 1: 'admin_level' not recognised as an available field. > Tue Apr 1 19:04:57 2014: CPLError: 'admin_level' not recognised as an > available field. > ERROR 1: 'admin_level' not recognised as an available field. > Tue Apr 1 19:04:57 2014: CPLError: 'admin_level' not recognised as an > available field. > ERROR 1: 'admin_level' not recognised as an available field. > Tue Apr 1 19:04:57 2014: OSM: OGR_INTERLEAVED_READING = 1 > Tue Apr 1 19:04:57 2014: CPLError: 'admin_level' not recognised as an > available field. > ERROR 1: 'admin_level' not recognised as an available field. > Tue Apr 1 19:04:57 2014: OSM: Got bounds : minx=-180.000000, > miny=-90.000000, maxx=180.000000, maxy=90.000000 > Tue Apr 1 19:04:57 2014: OSM: For layer points, new max size is 128 > Tue Apr 1 19:04:57 2014: OSM: For layer points, new max size is 320 > Tue Apr 1 19:04:57 2014: OSM: For layer points, new max size is 608 > Tue Apr 1 19:04:57 2014: OSM: For layer points, new max size is 1040 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 1688 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 2660 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 4118 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 6305 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 9585 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 14505 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 21885 > Tue Apr 1 19:04:58 2014: OSM: For layer points, new max size is 32955 > Tue Apr 1 19:05:01 2014: OSM: all_tags/other_tags field truncated for > feature 249399339 > Tue Apr 1 19:05:01 2014: OSM: For layer points, new max size is 49560 > Tue Apr 1 19:05:03 2014: OSM: For layer points, new max size is 74468 > Tue Apr 1 19:05:06 2014: OSM: > /vsimem/osm_importer/osm_temp_0x1d4a570.sqlite too big for RAM. > Transfering it onto disk in /tmp/osm_tmp1067730688_0 Tue Apr 1 19:05:20 > 2014: OSM: For layer points, new max size is 111830 Tue Apr 1 19:05:26 > 2014: CPLError: Too many features have accumulated in points layer. Use > OGR_INTERLEAVED_READING=YES mode > ERROR 1: Too many features have accumulated in points layer. Use > OGR_INTERLEAVED_READING=YES mode > Tue Apr 1 19:05:26 2014: OGR: > ReleaseDataSource(/go_video/admin-levels.pbf/0x1d4a570) on unshared > datasource! Deleting directly. > Tue Apr 1 19:05:26 2014: OSM: Number of bytes read in file : 143840051 > Tue Apr 1 19:05:26 2014: OGR: > ReleaseDataSource(/tmp/out/world1.shp/0x19a93d0) on unshared datasource! > Deleting directly. > Tue Apr 1 19:05:26 2014: GDAL: In GDALDestroy - unloading GDAL shared > library. """ > > However, using ogr2ogr utility on the command line works like a charm. If I > convert PBF->OSM, script will work, but exported data is missing some of > the features. I can provide test data and the script if required. > > Thank you, > > Dražen > > > [0] https://wiki.openstreetmap.org/wiki/Osmconvert > [1] https://wiki.openstreetmap.org/wiki/Osmfilter > _______________________________________________ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Geospatial professional services http://even.rouault.free.fr/services.html _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
