Hi Even, We - Martin Boos (new in Cc) and I - are definitely thinking about implementing an option for spatial bounds. Youmay know e.g. this WFS option of QGIS? „[_] Only request features overlapping the view extent“
What would you prefer: The same behaviour as for the WFS provider, or taking a fixed bounds which aren't following users pan/zoom actions (unless the user explicitly resets the bounds)? :Stefan @Blake: We are very interested in you opinion regarding handling of keys "crs" and "bounds" now as well as part of the future TileJSON spec. ---------- Forwarded message ---------- From: Even Rouault <even.roua...@spatialys.com> Date: 2017-04-17 11:44 GMT+02:00 Subject: Re: [gdal-dev] Vector Tiles in OGR To: Stefan Keller <sfkel...@gmail.com> Cc: Gdal-Dev <gdal-dev@lists.osgeo.org>, Flippmoke <flippm...@gmail.com>, Even Rouault <even.roua...@mines-paris.org>, petr.pri...@klokantech.com Hi Stefan, > P.S. @Even: Any news on a GDAL/OGR MVT reader? No activity on this front as far as I know. There was a possible funding opportunity recently for a project that would have involved integating such a driver with QGIS, but the odds for it to come true seems to be very low according to the latest updates I got. And your plugin might actually solve the needs of this project ! Quick question: from my test of the plugin, I see it converts the whole mbtiles and optionally merges geometry chunks of the same feature spread over several tiles into a single geometry, right ? In the preliminary discussions for the project I mentionned, it was not clear if there was a need for doing just a decoding & rendering of tiles that intersect the window of interest (could make sense if using very large mbtiles or using remote tiles), instead of a whole mbtiles conversion. Partial decoding, with merging of the tiles intersecting the window of interest, seems to be a bit at odds with the (implicit ?) assumptions of the QGIS vector layers that assume that the geometry of a feature will always be the same when it is returned, and not clamped to the window of interest. Even > > 2016-01-31 23:22 GMT+01:00 Flippmoke <flippm...@gmail.com>: > > All, > > > > The first thing that is going to be required likely is a good generic c++ > > library for encoding and decoding vector tiles IMO. > > > > I have spent a lot of time working on developing the mapnik vector tile > > library to be a solid implementation, especially around all the work done > > for 2.0 of Mapbox vector tile specification. I would love to eventually > > help make this a generalized implementation so that it is portable to a > > library like GDAL. > > > > We have talked about doing something like this at Mapbox but will require > > a good deal of time, so it hasn't been pushed forward yet. The other > > issues that come to mind to me is that some of the libraries that mapnik > > vector tile depends upon most likely need good custom implementations if > > we wanted to make it more generic. > > > > The reason for this is that I would prefer for it to be a header only > > library with no dependencies. Adding more dependencies for GDAL seems > > like it could be a mess. > > > > Blake Thompson > > > >> On Jan 31, 2016, at 10:10 AM, Stefan Keller <sfkel...@gmail.com> wrote: > >> > >> Hi Petr, Blake, Even and everybody > >> > >> I'm (besides Petr) the other maintainer of OSM2VectorTiles. > >> I'd already asked Even a similar question. > >> I'm advising now an intern to implement such a MVT Vector Tiles reader > >> client in Python if possible as part of a QGIS plugin. > >> And I'm interested not to do redundant implementations. > >> Are there any news on this matter? > >> > >> :Stefan > >> > >> Prof. at HSR and leader of Geometa Lab > >> > >> @Even: Regarding OGR MVT development can you send to me (directly) a > >> contact name of the group which won the bidding? > >> > >> 2016-01-06 15:24 GMT+01:00 Petr Pridal <petr.pri...@klokantech.com>: > >>> Hi everybody, > >>> > >>> thanks for the comments. Regarding the technical details: > >>>> Each Tile is its own discrete dataset therefore the hardest part would > >>>> be > >>>> merging of features into the original features across tiles. You could > >>>> use > >>>> some complicated merging technique, but that could be very expensive. > >>> > >>> The driver could require "id" parameter, which specifies name of an > >>> attribute with unique identifier on the features shared across the > >>> tiles. > >>> Both MapBox vector tiles and OSM2VectorTiles tiles have an unique > >>> non-zero > >>> "osm_id" in place. The first version of driver could read only features > >>> with such ID. The identifier does not have to be always filled, but > >>> often it is, especially in deeper zoom levels derived purely from > >>> OpenStreetMap - but yeh, a completely general stitching would be much > >>> harder. > >>> > >>> The clipping + merging of shapes itself can be implemented with > >>> underlaying > >>> GEOS library functions. > >>> > >>> BTW It is possible to examine vector tiles data in pure JavaScript X-Ray > >>> viewer (no WebGL support on browser required) made in OpenLayers3: > >>> http://klokantech.github.io/ol3-sandbox/vector/xray.html > >>> > >>> or with WebGL-enabled webbrowser with higher performance MapBox GL JS > >>> powered X-Ray: > >>> http://klokantech.github.io/mapbox-gl-js-offline-example/xray.html > >>> > >>> A debug viewer for each vector tile (made in OL3) decoded in pure > >>> JavaScript: > >>> http://klokantech.github.io/ol3-sandbox/vector/tile-inspector.html > >>> > >>> The OGR driver could be also practical for the unsimplified / > >>> ungeneralised > >>> MBTiles with OSM data available at: > >>> http://osmlab.github.io/osm-qa-tiles/ > >>> > >>>> If you wanted to simply view vector tile data it could get confusing as > >>>> well since most vector tiles need a buffer when they are used for > >>>> visualization programs. This is important for things like labeling on > >>>> maps > >>>> etc. Therefore, you could get a large amount of overlapping data on > >>>> tiles if you loaded them all in at once. > >>> > >>> Clipping on border of each tile must be applied before stitching the > >>> vector > >>> features together - so the buffer is removed. Buffer is there only for > >>> visualisation and labeling. > >>> > >>>> As far as writing a simple decoder -- the mapnik-vector-tile decoder > >>>> isn't > >>>> currently able to decode vector tiles into a form not supported by > >>>> mapnik. > >>>> Therefore, OGR would not be able to easily use this. > >>> > >>> Correct. It can't be used directly (GDAL can't link against), but the > >>> relevant code can be extracted and reused directly. For basic decoding > >>> critical is the Protobuf library (which is already in GDAL) and > >>> https://github.com/mapbox/mapnik-vector-tile/blob/master/proto/vector_ti > >>> le.proto + the code for converting pixel coordinates in features to > >>> relevant geo coordinates and GEOS-based merging of features. > >>> > >>>> We plan to write a generalized encoder and decoder eventually so that > >>>> any > >>>> other library can plugin with out the mapnik requirement. > >>> > >>> This would be amazing. > >>> > >>>> If you have any questions specifically about Mapnik Vector Tile or the > >>>> Mapbox Vector Tile Specification feel free to ask me. I would note that > >>>> you > >>>> guys will want to update your vector tiles once the V2 specification > >>>> changes are into Mapnik-Vector-Tile. > >>> > >>> Cool! Thanks. Our world tiles are generated via Docker containers on a > >>> cluster of computers - using Mapnik via Tilelive directly - so upgrades > >>> on > >>> Mapnik-Vector-Tile will propagate to the open-source rendering software. > >>> I expect V2 spec will mean also new MapBox Streets V7, right? > >>> > >>>> I see you mention MVT in MBTiles. Is it a recognized practice ? I don't > >>>> see > >>>> mention of that neither in the MVT spec ( > >>>> https://github.com/mapbox/vector- > >>>> tile-spec/tree/master/2.0 ) nor the MBTiles one ( > >>>> https://github.com/mapbox/mbtiles-spec/blob/master/1.2/spec.md ), > >>>> although > >>>> I > >>>> guess it is just a matter of storing a MVT blob in the tile_data > >>>> column. > >>> > >>> Yes. MapBox Studio Classic generates such MBTiles with Vector Tiles > >>> inside - and Mapnik reads them directly when rasterising. Storing the > >>> vector tiles in SQLite make sense - and it simplifies deployment, and > >>> you can then easily have complete world on each node of a cluster of > >>> tileservers. If you don't want to upgrade your base map too often, this > >>> approach is pretty efficient and removes the centralised database as > >>> typical bottleneck. > >>> > >>> I did some tests on vector tiles vs raster in MBTiles and documented > >>> these > >>> in README.md at and https://github.com/klokantech/vector-tiles-sample > >>> before we started to work on OSM2VectorTiles project. > >>> There are also sample data and offline MapBox GL JS viewer in that repo. > >>> Vector tiles can be hosted also "unpacked", just as files in folders on > >>> a > >>> standard web server - exactly like GDAL2Tiles or MapTiler.com does by > >>> default for raster tiles. > >>> > >>> If OGR driver is implemented, the primary source should be probably > >>> reading > >>> from an URL via curl. > >>> > >>> Reading PBFs from an MBTiles (or another SQLite) is just the most > >>> practical > >>> portable alternative to it. > >>> > >>> Technically people may store the tiles in different ways - for example > >>> Wikimedia guys (http://maps.wikimedia.org/ and > >>> https://github.com/kartotherian/) use Cassandra for tile storage, I > >>> guess > >>> MapBox internally is also on a different storage for production servers > >>> ... > >>> for transfers and streaming tilesets there is the - but tiles are > >>> ALWAYS > >>> exposed via HTTP. > >>> > >>> Regards, > >>> > >>> Petr > >>> -- > >>> Petr Pridal, Ph.D. > >>> CEO > >>> > >>> Klokan Technologies GmbH > >>> Hofnerstrasse 98, 6314 Unterageri, Switzerland > >>> Tel: +41 (0)41 511 26 12 > >>> Email: i...@klokantech.com > >>> Web: http://www.klokantech.com/ > >>> > >>> _______________________________________________ > >>> gdal-dev mailing list > >>> gdal-dev@lists.osgeo.org > >>> http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Spatialys - Geospatial professional services http://www.spatialys.com _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
