Re: [gdal-dev] Streaming Parser for OGR GeoJSON Driver

2016-01-29 Thread Even Rouault
> > Perhaps it works because I am only appending to a single layer that has the > same schema throughout? Yes > I did test this, but the performance was indeed slower. Is there a way for > me to specify the schema before-hand and avoid a full first pass? > > Perhaps for a driver implementation

Re: [gdal-dev] Streaming Parser for OGR GeoJSON Driver

2016-01-28 Thread Daniel Fenton
Thanks for your reply Even. Very helpful! >I'm not a JS programmer, nevertheless I tried to understand https://github.com/koopjs/GeoXForm/blob/master/src/lib/vrt.js, and you seem to group GeoJSON Feature objects by batch of 5000 (*) , put them in a temp .json file, and assemble all the JSon files

Re: [gdal-dev] Streaming Parser for OGR GeoJSON Driver

2016-01-27 Thread Even Rouault
Hi, > > I’m curious if anyone has ideas or advice on how to use a streaming parser > in the OGR GeoJSON driver. > A streaming parser, or at least something not requiring full ingestion in memory of a geojson file, is something that would indeed solve issues that people run into with the curre

[gdal-dev] Streaming Parser for OGR GeoJSON Driver

2016-01-26 Thread Daniel Fenton
Hi all, I’m curious if anyone has ideas or advice on how to use a streaming parser in the OGR GeoJSON driver. My use-case is that I need to convert arbitrarily-sized streams of geojson into other formats (e.g. Csv, shapefile, kml, etc). My current strategy is to first partition the GeoJSON in