Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Andreas Oxenstierna
If it is 20 mill. coordinates, it could be faster than 80 minutes, I guess in the region of 10 minutes. I would: 1 Combine 3+4 to something like (I assume ST_Multi is not needed) - this avoids data duplication and one unnecessary transaction commit    SELECT randfield, (st_dump(ST_Union(f.geom)

Re: [gdal-dev] Docker images for development/testing?

2018-07-16 Thread Kurt Schwehr
I might suggest going with debian testing to get reasonably new dependent libs (if that matters to you). I've been using it for 6 months for my desktop and it has worked well for me. On Mon, Jul 16, 2018 at 2:07 PM, Even Rouault wrote: > > > > I made a very rough proof of concept out of the tru

Re: [gdal-dev] Docker images for development/testing?

2018-07-16 Thread Even Rouault
> > I made a very rough proof of concept out of the trusty_clang travis > scripts... > Thoughts? Comments? Good initiative. As far as I'm concerned, the scripts/setdevenv.sh on my native Linux env does the job, but that indeed requires to have all the dependencies right. > > Without other st

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Paul Meems
I took the advice of Andreas and converted my code to using PostGIS. And the speed difference is enormous. The commands I've used: // Import shapefile into PostGIS: ogr2ogr -f PostgreSQL PG:"host=localhost user=..." fishnet.shp -gt unlimited -lco GEOMETRY_NAME=geom -a_srs "EPSG:28992" // Add rando

Re: [gdal-dev] Docker images for development/testing?

2018-07-16 Thread Howard Butler
On 7/16/18 8:51 AM, Robert Coup wrote: > Suggestion: > > 1. docker images with all the build/test/library dependencies already > installed — publish them so getting the right dependencies & environment is > only a download. > > > Thoughts? Comments? PDAL uses Alpine+Docker for its TravisCI builds

[gdal-dev] Docker images for development/testing?

2018-07-16 Thread Robert Coup
Hi All, For me at least, maintaining working dev environments for GDAL can be frustrating... there are a lot of dependencies, platforms, and a huge number of config options. Let alone switching branches for backports/etc. I typically use OSX as a desktop and linux environments running under that v

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Andrew C Aitchison
On Mon, 16 Jul 2018, Paul Meems wrote: Thanks, Jon for your suggestion of GeoPandas. Unfortunately, I'm not allowed to use new external dependencies. Some timing: 1,677 shapes --> 0.3s 4,810 shapes --> 1.8s 18,415 shapes --> 21.4s 72,288 shapes --> 5min, 54s 285,927 shapes --> 25m 1,139,424 s

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Andreas Oxenstierna
ST_Union in PostGIS should scale better than SQLite. ST_Dump gives you singlepart geometries. Best Regards Andreas Oxenstierna > 16 juli 2018 kl. 10:53 skrev Paul Meems : > > Thanks, Jon for your suggestion of GeoPandas. > Unfortunately, I'm not allowed to use new external dependencies. >

Re: [gdal-dev] Dissolve large amount of geometries

2018-07-16 Thread Paul Meems
Thanks, Jon for your suggestion of GeoPandas. Unfortunately, I'm not allowed to use new external dependencies. I tried doing all steps in an SQLite file instead of using several intermediate shapefiles. And I had some good results, so I created a script dissolving an increasingly higher number of