Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Andreas Oxenstierna
If it is 20 mill. coordinates, it could be faster than 80 minutes, I guess in the region of 10 minutes. I would: 1 Combine 3+4 to something like (I assume ST_Multi is not needed) - this avoids data duplication and one unnecessary transaction commit    SELECT randfield, (st_dump(ST_Union(f.geom)

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Paul Meems
I took the advice of Andreas and converted my code to using PostGIS. And the speed difference is enormous. The commands I've used: // Import shapefile into PostGIS: ogr2ogr -f PostgreSQL PG:"host=localhost user=..." fishnet.shp -gt unlimited -lco GEOMETRY_NAME=geom -a_srs "EPSG:28992" // Add rando

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Andrew C Aitchison
On Mon, 16 Jul 2018, Paul Meems wrote: Thanks, Jon for your suggestion of GeoPandas. Unfortunately, I'm not allowed to use new external dependencies. Some timing: 1,677 shapes --> 0.3s 4,810 shapes --> 1.8s 18,415 shapes --> 21.4s 72,288 shapes --> 5min, 54s 285,927 shapes --> 25m 1,139,424 s

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Andreas Oxenstierna
ST_Union in PostGIS should scale better than SQLite. ST_Dump gives you singlepart geometries. Best Regards Andreas Oxenstierna > 16 juli 2018 kl. 10:53 skrev Paul Meems : > > Thanks, Jon for your suggestion of GeoPandas. > Unfortunately, I'm not allowed to use new external dependencies. >

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Paul Meems
Thanks, Jon for your suggestion of GeoPandas. Unfortunately, I'm not allowed to use new external dependencies. I tried doing all steps in an SQLite file instead of using several intermediate shapefiles. And I had some good results, so I created a script dissolving an increasingly higher number of

Re: [gdal-dev] Dissolve large amount of geometries

2018-06-29 Thread Jon Morris
lans for a Dissolve method on OGRLayer? Jon From: gdal-dev On Behalf Of Paul Meems Sent: 29 June 2018 09:51 To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Dissolve large amount of geometries Thanks Even for the suggestion. We already thought about polygonizing the raster. But then I would

Re: [gdal-dev] Dissolve large amount of geometries

2018-06-29 Thread Paul Meems
Thanks Even for the suggestion. We already thought about polygonizing the raster. But then I would have less control about where the vector geometry is created. The squares/rectangle need to align with the tractor tracks/path. The scenario I described is the first version: creating a fishnet. The

Re: [gdal-dev] Dissolve large amount of geometries

2018-06-28 Thread Even Rouault
On jeudi 28 juin 2018 12:53:27 CEST Paul Meems wrote: > Hi list, > > I've been working on this for months (off and on) and still no satisfying > outcome. > Either the process takes too long (multiple hours) or the result has > invalid geometries. > > I want to try a different angle now. Instead o

[gdal-dev] Dissolve large amount of geometries

2018-06-28 Thread Paul Meems
Hi list, I've been working on this for months (off and on) and still no satisfying outcome. Either the process takes too long (multiple hours) or the result has invalid geometries. I want to try a different angle now. Instead of asking technical questions I want to explain what I try to do. Hopef