[gdal-dev] FlatGeobuf Layer vs Feature Columns

2025-03-06 Thread Patrick Young via gdal-dev
Hello, I translated some geojson to flatgeobuf where the geojson features have a variable number of properties (e.g. some features have a bunch of translations for the name of a place pulled from OSM). I noticed in the ogrinfo output that while the summary info has all the fields possible, name:

Re: [gdal-dev] "Regular" SQLite DB Format

2024-10-25 Thread Patrick Young via gdal-dev
Ah thanks, yeah I see now with a '-dsco METADATA=NO' option ogrinfo doesn't recognize the output geometry column as geometry. On Fri, Oct 25, 2024 at 3:46 PM Even Rouault wrote: > > Le 25/10/2024 à 23:40, Patrick Young via gdal-dev a écrit : > > Hello, > > >

[gdal-dev] "Regular" SQLite DB Format

2024-10-25 Thread Patrick Young via gdal-dev
Hello, I noticed in the docs https://gdal.org/en/latest/drivers/vector/sqlite.html#regular-sqlite-databases that there is some support for treating a plain sqlite db as a vector format. I've been dumping some vector data into a table with a geometry column full of WKB plus some properties, but I

[gdal-dev] Most Efficient MVT Creation

2024-10-01 Thread Patrick Young via gdal-dev
Hi, I've been playing around a little with the MVT driver and noticed in the test suite, features are written first via the Memory driver and then gdal.VectorTranslate is used to create the MVTs. Is there any advantage to doing this vs writing features directly to a MVT datasource? From what I c

Re: [gdal-dev] Reading PMTiles With Missing Tiles

2024-02-23 Thread Patrick Young via gdal-dev
uault wrote: > Hi, > > this is likely a bug. I would need access to the file to investigate, > ideally through /vsicurl, /vsis3, etc. . I don't have storage to host > 300 GB. And possibly try to reduce the file size by removing upper zoom > levels > > Even > >

[gdal-dev] Reading PMTiles With Missing Tiles

2024-02-22 Thread Patrick Young via gdal-dev
Hello, I've been exploring the pmtiles driver a bit, what a great way to read giant blobs of tiles off of cloud storage, really impressed by how fast this is. I've been encountering a "Non increasing tile_id" when attempting to read from coarser zoom levels for a large (~300GB) pmtile file: http

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

2023-07-07 Thread Patrick Young
Thanks Even! On Mon, Jul 3, 2023, 3:11 AM Even Rouault wrote: > Hi Patrick, > > blocking error fixed per https://github.com/OSGeo/gdal/pull/8040 > > and the one related to esriFieldTypeDate in > https://github.com/OSGeo/gdal/pull/8039 > > Even > > Le 28/06/2023

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

2023-06-28 Thread Patrick Young
Area" FLOAT,OGR_STYLE VARCHAR HIDDEN,"SHAPE" BLOB_MULTIPOLYGON_XY_2263, OGR_NATIVE_DATA VARCH AR, OGR_NATIVE_MEDIA_TYPE VARCHAR)) In the 3.5+ versions, sqlite is angry about the INTEGER_INT16_BEGIN_DOMAIN_NAME_64526F6164626564_END_DOMAIN_NAME On Wed, Jun 28, 2023 at 9:18 AM Patrick Young &

Re: [gdal-dev] ogr2ogr sqlite dialect and fgb format

2023-06-28 Thread Patrick Young
grinfo dataset.gdb > > On 6/28/23 08:18, Patrick Young wrote: > > Hi, > > > > I'm playing around with the gdb dataset > > > > https://data.cityofnewyork.us/Transportation/NYC-Planimetrics/wt4d-p43d > > <https://data.cityofnewyork.us/Transportation/NYC-Plan

[gdal-dev] ogr2ogr sqlite dialect and fgb format

2023-06-28 Thread Patrick Young
Hi, I'm playing around with the gdb dataset https://data.cityofnewyork.us/Transportation/NYC-Planimetrics/wt4d-p43d and getting errors starting with gdal 3.5 when I try to use the sqlite dialect: ogr2ogr -f GeoJSONSeq -sql "select shape from roadbed where source_id = 13350003311" -dialect sqlit

[gdal-dev] arm64 Impressions?

2021-09-30 Thread Patrick Young
Hi, I was curious if anyone had any thoughts about GDAL and arm64. I saw that Even had been playing with arm64 a bit https://lists.osgeo.org/pipermail/gdal-dev/2021-July/054418.html and AWS talks a lot about the efficiency of their Gravitron stuff. Does anyone have thoughts about arm64 and GDA

Re: [gdal-dev] Enabling default compression for COG ?

2021-09-30 Thread Patrick Young
Another option would be to emit a warning/suggestion when left unset. An added benefit would be that folks possibly self discover that they can control the compression scheme. P On Thu, Sep 30, 2021, 7:53 AM Vincent Sarago wrote: > +1 > I think it’s reasonable to have a default compression. In

Re: [gdal-dev] gdalwarp alpha band resampling bug

2021-09-17 Thread Patrick Young
One more data point; another colleague who is on a mac but built gdal himself instead of using brew does not see the artifact, so I suppose the lesson is, build your own gdal! Best, P On Fri, Sep 17, 2021 at 8:51 PM Patrick Young < patrick.mckendree.yo...@gmail.com> wrote: > Hi Even, &

Re: [gdal-dev] gdalwarp alpha band resampling bug

2021-09-17 Thread Patrick Young
green and blue bands is < 255). But I > suspect this could be related to a pre-existing composite-bl.tif file you > might have. gdalwarp doesn't overwrite by default the output image. It > blends into it. If you add -overwrite, you should get a reasonable output. > Le

[gdal-dev] gdalwarp alpha band resampling bug

2021-09-17 Thread Patrick Young
Hello, I've found behavior in gdalwarp related to compositing an image with a variable alpha band. Under bilinear resampling, the alpha compositing in areas where the alpha band is between 0 and 255 results in completely white pixels. To reproduce, I've first grabbed a rgba image from the test s

Re: [gdal-dev] On Internal PROJ When Building

2021-08-26 Thread Patrick Young
t proj > 6.3.1 from the Ubuntu 20.04 distro, whereas the GDAL we build will link > against the PROJ built in the image, so without that trick, crashes would > happen. > > Even > Le 26/08/2021 à 17:21, Patrick Young a écrit : > > Hi, > > I was curious how rel

[gdal-dev] On Internal PROJ When Building

2021-08-26 Thread Patrick Young
Hi, I was curious how relevant building an internal PROJ library is these days as described in this note https://trac.osgeo.org/gdal/wiki/BuildingOnUnixGDAL25dev I noticed GDAL's docker builds still do this. Are there some common dependencies that still use the older PROJ? Many thanks! Patrick

Re: [gdal-dev] Unable to convert NTF to png

2021-06-29 Thread Patrick Young
What is the bit depth of the NTF? You might need to scale it to an 8bit range when converting it, check out the -scale option flag for gdal_translate https://gdal.org/programs/gdal_translate.html#cmdoption-gdal_translate-scale On Tue, Jun 29, 2021 at 3:33 PM Aparna Nimbalker via gdal-dev < gdal-

Re: [gdal-dev] Python GDAL split polyline by intersection

2021-04-23 Thread Patrick Young
You might check out the Python library Shapely (which wraps GEOS), it has a split method in it: https://shapely.readthedocs.io/en/stable/manual.html#splitting P On Fri, Apr 23, 2021 at 9:59 AM Chang Liao wrote: > Hi, GDALers, > > I have a GDAL Python API question, i want to find some function

Re: [gdal-dev] Gdal.Open on S3 using IAM roles

2021-04-14 Thread Patrick Young
WS_DEFAULT_PROFILE: { > Gdal.GetConfigOption("AWS_DEFAULT_PROFILE", "NotFound")} "); > _awsBucketName = > configStore.GetValueString("AWS_DESIGNIMPORT_BUCKET_NAME"); > } > > and this is the open which times out: > public string GetMeta

Re: [gdal-dev] Gdal.Open on S3 using IAM roles

2021-04-14 Thread Patrick Young
that none of the following are set >> AWS_PROFILE; AWS_ACCESS_KEY_ID; AWS_SECRET_ACCESS_KEY; AWS_SESSION_TOKEN; >> AWS_NO_SIGN_REQUEST; AWS_DEFAULT_PROFILE >> >> >> -- >> *Jeannie May* >> Senior Software Engineer >> 11 Birmingham Drive, Christchurch | 963 5

Re: [gdal-dev] Gdal.Open on S3 using IAM roles

2021-04-13 Thread Patrick Young
See https://aws.amazon.com/premiumsupport/knowledge-center/iam-assume-role-cli/ , they describe how to assume a role and set the usual AWS_* environment variables that GDAL should pick up. There's discussion on vsis3 related stuff (e.g. authentication) here: https://gdal.org/user/virtual_file_syst

Re: [gdal-dev] COG + javascript + openlayers

2021-04-13 Thread Patrick Young
There is a COG mailing list (not very active from the looks of it): https://lists.osgeo.org/mailman/listinfo/cog You might ping Vincent https://github.com/vincentsarago too! P On Tue, Apr 13, 2021 at 2:31 AM Javier Jimenez Shaw wrote: > Maybe this channel is not the best place for this questi

Re: [gdal-dev] Question Regarding Custom gdal_translate

2021-03-19 Thread Patrick Young
FWIW check out https://github.com/OSGeo/gdal/blob/master/autotest/utilities/test_gdal_translate.py https://github.com/OSGeo/gdal/blob/master/autotest/utilities/test_gdal_translate_lib.py On Fri, Mar 19, 2021 at 3:35 AM Andrew C Aitchison wrote: > On Fri, 19 Mar 2021, mat data wrote: > > > I am

Re: [gdal-dev] What is lost when converting 12 bit imagery to 8 bit?

2021-03-17 Thread Patrick Young
I would guess you usually see 8bit RGB images because that is what your monitor can display. What is lost is a deeper question, per channel you have to squeeze the original [0 - 4095] pixel value range per channel down to [0 -255], and there are lots of ways to do it. The problem is sometimes ca

Re: [gdal-dev] validate_cloud_optimized_geotiff.py returns invalid

2021-01-29 Thread Patrick Young
Sorry gdal_translate! On Fri, Jan 29, 2021 at 2:34 PM Patrick Young < patrick.mckendree.yo...@gmail.com> wrote: > What happens if you run it through "gdalwarp -of COG in.tif out.tif"? > > On Fri, Jan 29, 2021 at 10:05 AM Javier Jimenez Shaw > wrote: > >> Hi

Re: [gdal-dev] validate_cloud_optimized_geotiff.py returns invalid

2021-01-29 Thread Patrick Young
What happens if you run it through "gdalwarp -of COG in.tif out.tif"? On Fri, Jan 29, 2021 at 10:05 AM Javier Jimenez Shaw wrote: > Hi all > > I am generating a COG file with GDAL 3.2.0 (compiled in my Ubunut 18.04) > in C++ > Nothing strange, just > > GDALDriver* driver = GetGDALDriverManager()

Re: [gdal-dev] mosaicking files with partial overlap and binary mask

2021-01-27 Thread Patrick Young
Hi Daniele, Even added this functionality to VRTs just recently, see https://lists.osgeo.org/pipermail/gdal-dev/2020-December/053178.html I think it is slated for GDAL 3.3 but you could always build the bleeding edge. Patrick On Wed, Jan 27, 2021 at 3:50 AM Daniele Romagnoli < daniele.romagn..

Re: [gdal-dev] Slow speed in remote reading

2020-11-02 Thread Patrick Young
You can check how a S2 COG compares at https://registry.opendata.aws/sentinel-2-l2a-cogs/ You should hopefully see a lot less network requests when accessing a sub block directly. P On Mon, Nov 2, 2020 at 7:58 AM Alexander Verbitsky wrote: > Hi All, I am trying to implement remote reading sm

Re: [gdal-dev] Source SRS is a compound CRS but lacks +geoidgrids

2020-06-16 Thread Patrick Young
Thanks Even, really helpful! Patrick On Tue, Jun 16, 2020 at 3:45 PM Even Rouault wrote: > On mardi 16 juin 2020 15:17:54 CEST Patrick Young wrote: > > > Is this a special case for EPSG:5773, or is it a bad idea to express the > > > vertical datum via EPSG codes in genera

Re: [gdal-dev] Source SRS is a compound CRS but lacks +geoidgrids

2020-06-16 Thread Patrick Young
Is this a special case for EPSG:5773, or is it a bad idea to express the vertical datum via EPSG codes in general? Patrick On Tue, Jun 16, 2020 at 2:34 PM Even Rouault wrote: > On mardi 16 juin 2020 20:22:57 CEST Comer, Alex wrote: > > > Hello, I'm having trouble using gdalwarp to adjust the he

Re: [gdal-dev] Python Wheels for gdal

2020-01-31 Thread Patrick Young
Here is how I build them for Linux, taken from Sean's work on rasterio. https://github.com/youngpm/gdalmanylinux On Fri, Jan 31, 2020, 2:58 AM Christoph Paulik wrote: > Thank you all for the input. > > My initial motivation was that I would like `pip install gdal` to just > work. > > I like

Re: [gdal-dev] Querying many raster pixels efficiently from Python

2020-01-23 Thread Patrick Young
Putting something like mapserver/mapcache in front of your requests might work, if I understand correctly. We serve COGs out of s3 via WMS like this and the performance is pretty nice. See https://github.com/pedros007/mapserver-docker for some discussion on such a setup. Best, Patrick On Thu

Re: [gdal-dev] gdalwarp, by reading .RPB file

2020-01-15 Thread Patrick Young
th DEM to orthorectify the > image? > Thanks a lot. > Elmir > > ‐‐‐ Original Message ‐‐‐ > On Wednesday, January 15, 2020 11:13 AM, Patrick Young < > patrick.mckendree.yo...@gmail.com> wrote: > > Hi Elmir, > > When you run gdalinfo against the tif, it shoul

Re: [gdal-dev] Recommended process for running a intersect test between a raster and a simple vector

2020-01-14 Thread Patrick Young
One trick I like regarding accessing overview levels is to use the open option to snag a handle direct to the overview level (see Even's post http://erouault.blogspot.com/2014/10/warping-overviews-and-warped-overviews.html); then you can run polygonize against a much smaller dataset. If it's a COG

Re: [gdal-dev] Python Bindings and Closed Datasets

2019-10-31 Thread Patrick Young
After some more investigation, I've managed to put together the script below that manifests the issue on my machine (8 core, Ubuntu 19.04, GDAL 2.4.0, Python 3.7). Where I first observed this was in using 8 band WV2 imagery, so that's what the complicated input image is modeled after. Here's an ex

[gdal-dev] Python Bindings and Closed Datasets

2019-10-30 Thread Patrick Young
Hi all, I've been experiencing some behavior using the GDAL python bindings where I am occasionally seeing what appears to be random blocks of the tiff being unwritten in geotiffs I've pushed to S3. a small block(s) in one of the bands will be all zeros while everywhere else is good. My setup is

Re: [gdal-dev] Ellipsoidal length of a line

2019-06-14 Thread Patrick Young
Not exactly what you want, but you can do this with PostGIS by casting your geometry to the geography type: https://postgis.net/workshops/postgis-intro/geography.html On Fri, Jun 14, 2019 at 10:26 AM Atle Frenvik Sveen wrote: > Hi! > > Not sure if would want to use gdal for this task*, but take

Re: [gdal-dev] Extracting Subsampled Mask Band from VRT Returning 1st Band Rather Than Mask Band

2019-05-15 Thread Patrick Young
clues! Patrick On Wed, May 15, 2019 at 11:17 AM Patrick Young < patrick.mckendree.yo...@gmail.com> wrote: > Hi all, > > I'm experiencing some strange behavior when extracting a mask band using > gdal_translate from a VRT vs the source GTiff at a reduced resolution.

Re: [gdal-dev] Extracting Subsampled Mask Band from VRT Returning 1st Band Rather Than Mask Band

2019-05-15 Thread Patrick Young
An additional data point; loading the VRT in QGIS, everything looks correct at full resolution (e.g. alpha as transparency in the right spot) but as you zoom out, alpha suddenly becomes the first band and the transparency is bonkers. On Wed, May 15, 2019 at 11:17 AM Patrick Young

[gdal-dev] Extracting Subsampled Mask Band from VRT Returning 1st Band Rather Than Mask Band

2019-05-15 Thread Patrick Young
Hi all, I'm experiencing some strange behavior when extracting a mask band using gdal_translate from a VRT vs the source GTiff at a reduced resolution. I have a 3 band, 8 bit JPEG compressed (YCbCR) RGB geotiff with an internal no data mask and with internal overviews. Making a VRT of this and r

Re: [gdal-dev] gdal.BuildVRT with an empty destination

2019-03-13 Thread Patrick Young
Thanks Even, this is awesome functionality! Definitely makes it even easier to work with VRTs in memory! On Wed, Mar 13, 2019 at 3:37 AM Even Rouault wrote: > On mardi 12 mars 2019 16:42:34 CET Patrick Young wrote: > > Hi all, > > > > I was browsing some GDAL test cases

[gdal-dev] gdal.BuildVRT with an empty destination

2019-03-12 Thread Patrick Young
Hi all, I was browsing some GDAL test cases and came across some tests like so: ds = gdal.BuildVRT('',...) aka the destination is an empty string. Is the returned dataset a purely in memory object in this case? I assume it is safe to use like any other dataset; some of the other VRT related te

Re: [gdal-dev] Construct a VRT such that is sees IMD, RPB, XML files?

2018-12-07 Thread Patrick Young
Well I used gdal_translate to make a VRT directly from the original TIF and it revealed the answer; Metadata! 226 501 ...and so on... On Fri, Dec 7, 2018 at 12:40 PM Patrick Young < patrick.mckendree.yo...@gmail.com> wrote: > Hi all, > > I'm got some code t

[gdal-dev] Construct a VRT such that is sees IMD, RPB, XML files?

2018-12-07 Thread Patrick Young
Hi all, I'm got some code that builds a VRT out of a bunch of image tiles. These tiles were originally chipped out of a DigitalGlobe 1B GeoTIFF that came with extra metadata, specifically gdalinfo reports that its finding a .IMD, .RPB, and .XML files along with the TIF itself, and gives me a nice

Re: [gdal-dev] Recommended OGR Driver For Very Large Vector Data

2018-08-07 Thread Patrick Young
Aug 5, 2018 at 11:15 AM, jratike80 < jukka.rahko...@maanmittauslaitos.fi> wrote: > Patrick Young wrote > > We also tried a GeoPackage, but it seemed a little slow to read in QGIS. > > Hi, > > Could you give some more details about your GeoPackage and what you mean >

Re: [gdal-dev] Recommended OGR Driver For Very Large Vector Data

2018-08-03 Thread Patrick Young
f you asked this on the MapServer-users list you would get a > whole lot of different opinions) > > But that is my initial recommendation, for your stated needs. > > -jeff > > > > -- > Jeff McKenna > MapServer Consulting and Training Services > https://gatewayge

[gdal-dev] Recommended OGR Driver For Very Large Vector Data

2018-08-03 Thread Patrick Young
Hi all, I've been trying to figure out a good format to store some very large vector data in. Basically, its a bunch of really dense polygons describing the cutlines that compose a mosaic. We've had reasonable success using zipped shapefiles, but on the larger guys we're exceeding the 2GB file s

[gdal-dev] GDAL Python Bindings via manylinux wheels

2016-10-25 Thread Patrick Young
Hi all, I've been working on an experiment to see if I can get the Python GDAL bindings packaged up as a "manylinux" wheel (see https://www.python.org/dev/peps/pep-0513/ and https://github.com/pypa/manylinux for more info) to see if I can run the GDAL utilities that are now exposed via the binding

[gdal-dev] gdalwarp -rpc with RPC_DEM default behavior not as described

2016-07-26 Thread Patrick Young
Hi all, We've been making some orthos with gdalwarp (gdal version 2.1.1) and encountering resampling errors in the output (gittery output in the lines in certain spots). Long story short, explicitly setting the "-et" option to "0" fixed it, but the docs for gdalwarp say the following (under the -

[gdal-dev] gdal.FileFromMemBuffer

2015-02-23 Thread Patrick Young
Hi all, I've been playing around a bit with the vsimem stuff in gdal using python, works great! Basically, I'm getting a json response from an ArcServer endpoint and I want to get out of the ESRI world as quick as I can. My question is, when I do something like this, r = requests.get(url, param