Re: [gdal-dev] array dim permute

2025-07-04 Thread Michael Sumner via gdal-dev
/OSGeo/gdal/pull/12700 will fix it > > Even > Le 04/07/2025 à 00:48, Michael Sumner via gdal-dev a écrit : > > I'm confused about the orientation of this in 2D mode 1485x1272 is > correct, with 122 bands. > > (requires earthdata auth) > > gdal raster info "/vs

[gdal-dev] array dim permute

2025-07-03 Thread Michael Sumner via gdal-dev
I'm confused about the orientation of this in 2D mode 1485x1272 is correct, with 122 bands. (requires earthdata auth) gdal raster info "/vsicurl/ https://obdaac-tea.earthdatacloud.nasa.gov/ob-cumulus-prod-public/PACE_OCI.20250501T234515.L2.SFREFL.V3_0.nc"; --output-format text | grep /rhos # S

[gdal-dev] blog post on Zarr vs COG

2025-05-23 Thread Michael Sumner via gdal-dev
https://element84.com/software-engineering/is-zarr-the-new-cog/ -- Michael Sumner Research Software Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.os

[gdal-dev] tmp files writing to object storage

2025-05-19 Thread Michael Sumner via gdal-dev
When I write a COG to object storage, I see the external overview being created in CPL_TMPDIR, but there is a .tif.tmp.tif file that then isn't cleaned up. This is on a custom endpoint. I see gdalwarp test_me.tif /vsis3/ideatest/sometest6.tif -of COG -co SPARSE_OK=YES -ts 14400 7200 -overwrite Cre

Re: [gdal-dev] replacing a dataset in C++

2025-05-10 Thread Michael Sumner via gdal-dev
Awesome, thanks so much. Cheers, Mike. On Sun, May 11, 2025 at 8:29 AM Even Rouault wrote: > > Le 11/05/2025 à 00:20, Michael Sumner via gdal-dev a écrit : > > Hello, I have a question about safely re-casting a dataset to another, I > want to use Multidim to re-orient an MDArr

Re: [gdal-dev] replacing a dataset in C++

2025-05-10 Thread Michael Sumner via gdal-dev
Ugh, sorry made a mistake in the code, fixed for osArrayName: auto poSrcDS = std::unique_ptr( GDALDataset::Open(osFilename, GDAL_OF_RASTER | GDAL_OF_VERBOSE_ERROR, aosAllowedDrivers.List(), aosOpenOptions.List(), nullptr)); // now open i

[gdal-dev] replacing a dataset in C++

2025-05-10 Thread Michael Sumner via gdal-dev
Hello, I have a question about safely re-casting a dataset to another, I want to use Multidim to re-orient an MDArray as a classic 2D raster. I was a bit surprised that this works (I just overwrite poSrcDS with the classic form): I'm passing in 'osFilename', 'osArrayName' and two dimension indexes

Re: [gdal-dev] Search This Mailing List?

2025-05-03 Thread Michael Sumner via gdal-dev
This archive is searchable, updates once a month I think: https://github.com/mdsumner/lists.osgeo.org On Sat, May 3, 2025, 01:25 Andrew Bell via gdal-dev < gdal-dev@lists.osgeo.org> wrote: > Hi, > > Is there an easy way to search the archives of this mailing list? > > Thanks, > > -- > Andrew Bel

Re: [gdal-dev] zarr urls need quoting

2025-03-26 Thread Michael Sumner via gdal-dev
Ok, thank you that's totally understandable and I will champion your choices here Appreciate the (fast) response very much Cheers, Mike On Thu, Mar 27, 2025, 04:28 Even Rouault wrote: > Michael, > > > > > > gdalmdiminfo > > ZARR:"/vsicurl/ > https://data.source.coop/dynamical/noaa-gefs-analysi

[gdal-dev] zarr urls need quoting

2025-03-26 Thread Michael Sumner via gdal-dev
Hi, I've explored the code a bit and can't figure out what's intended. When I try a ZARR url endpoint, it doesn't work from my first framing and I have to embed inner quotes around the dsn i.e. this works gdalmdiminfo "ZARR:\"/vsicurl/ https://data.source.coop/dynamical/noaa-gefs-analysis/v0.0.1

[gdal-dev] Building Scalable Mosaic Workflows with Flyte/UnionAI, GDAL, and Xarray (upcoming presentation)

2025-03-07 Thread Michael Sumner via gdal-dev
Sharing this for general interest "We demonstrate the use of Flyte, deployed by UnionAI, as a workflow orchestration system for constructing large-scale Xarray mosaics using GDAL’s new GTI driver." https://discourse.pangeo.io/t/pangeo-showcase-building-scalable-mosaic-workflows-with-flyte-unionai

Re: [gdal-dev] How to wrap a zarr in a zip and read it with vsizip?

2025-02-27 Thread Michael Sumner via gdal-dev
and the lack of overviews. I'm also a bit skeptical > about the Python ecosystem. My impression is that Zarr just tries (and > manages) to be a better NetCDF. > > Laurentiu > > > On Thu, Feb 27, 2025, at 06:06, Michael Sumner via gdal-dev wrote: > > Just clueing into why

Re: [gdal-dev] How to wrap a zarr in a zip and read it with vsizip?

2025-02-27 Thread Michael Sumner via gdal-dev
nice, I've been hitting that up with the multidim model and it works pretty well. (i'm still scratchy in the C++ but getting what I want out of it). We were checking out CMIP6 holdings too that look similarly impressive. M On Thu, Feb 27, 2025 at 5:27 PM Kurt Schwehr wrote: > Mike, > > That's

Re: [gdal-dev] How to wrap a zarr in a zip and read it with vsizip?

2025-02-26 Thread Michael Sumner via gdal-dev
Just clueing into why you might be working with this, have you seen this critique? (was a bit shocked to see that this is apparently going forward for Sentinel 2, let alone that it was even considered!) https://github.com/csaybar/ESA-zar-zip-decision Also glad to see a working example outlined t

Re: [gdal-dev] unsetting configs vs empty values

2025-02-19 Thread Michael Sumner via gdal-dev
into in practice. You're probably aware of this already, but > instead of: > > > export AWS_S3_ENDPOINT=... > > gdalmdiminfo ... > > unset AWS_S3_ENDPOINT > > gdalmdiminfo ... > > you can do: > > > AWS_S3_ENDPOINT=... gdalmdiminfo ... > > gdalmdimin

[gdal-dev] unsetting configs vs empty values

2025-02-18 Thread Michael Sumner via gdal-dev
This may well just be a hapless user-question, can we set configs to empty and expect that to mean "unset"? In a fresh session all is well (standard public bucket): gdalmdiminfo /vsis3/mur-sst/zarr Our non-standard public bucket can't be found of course, so set the endpoint: export AWS_S3_ENDPO

Re: [gdal-dev] Revert "Undo executive order 14172"

2025-02-14 Thread Michael Sumner via gdal-dev
Where? When it's a fascist state we're all at fault. I knew this wouldn't last but I absolutely supported it, silently, from the sidelines, frightened of its consequences... This is the slope. Good luck everyone, be brave On Fri, Feb 14, 2025, 20:07 Rahkonen Jukka via gdal-dev < gdal-dev@lists.o

[gdal-dev] no autodetect for ZARR output

2025-02-04 Thread Michael Sumner via gdal-dev
I can see that there's no calls to for the Zarr driver 'SetMetadataItem(GDAL_DMD_EXTENSION' or 'SetMetadataItem(GDAL_DMD_EXTENSIONS', but I wanted to ask about the behaviour. This makes sense to me: gdal_translate gdrivers/data/netcdf/srid.nc out.zarr -co COMPRESS=ZSTD Output driver not found.

Re: [gdal-dev] convert raster to raw table ?

2024-12-12 Thread Michael Sumner via gdal-dev
sn't do exactly what you want, but should be easy to modify to satisfy >> your requirements: >> https://github.com/OSGeo/gdal/blob/master/apps/gdal2ogr.c >> >> Even >> Le 12/12/2024 à 06:40, Michael Sumner via gdal-dev a écrit : >> >> Hi, I'd like to be

Re: [gdal-dev] convert raster to raw table ?

2024-12-12 Thread Michael Sumner via gdal-dev
: > https://github.com/OSGeo/gdal/blob/master/apps/gdal2ogr.c > > Even > Le 12/12/2024 à 06:40, Michael Sumner via gdal-dev a écrit : > > Hi, I'd like to be able to create a tabular version efficiently directly > from a raster. I can do this in scripting but is there a utility

[gdal-dev] convert raster to raw table ?

2024-12-11 Thread Michael Sumner via gdal-dev
Hi, I'd like to be able to create a tabular version efficiently directly from a raster. I can do this in scripting but is there a utility- or api- way to do this? I literally just want a Parquet file with a column for each band from a raster, with raw pixel values, no index, no coordinates, no rat

Re: [gdal-dev] NetCDF: how to read coordinates from "lon" and "lat" variables?

2024-12-03 Thread Michael Sumner via gdal-dev
You can't use a geotransform unless the data are actually regular. There's no saying whether the lon lat arrays describe a regular grid redundantly without inspecting them (it's also very common for formats to deliver lon lat coords when the data is actually a regular grid in a different crs, but i

Re: [gdal-dev] NetCDF: how to read coordinates from "lon" and "lat" variables?

2024-12-03 Thread Michael Sumner via gdal-dev
Try gdalinfo NETCDF:"out.input.nc":NO2 If you see those lon and lat subdatasets listed under "geolocation" then you can use gdalwarp to create a georeferenced dataset, automatically with gdalwarp sds.in out.tif or by specifying a grid with some of -tr,-te,-t_srs,-ts. If they don't appear they ca

Re: [gdal-dev] Mollweide to plate_carree

2024-10-28 Thread Michael Sumner via gdal-dev
I the first step you are a_ssigning moll but with a longlat ullr (extent ) a_ssigned. This isn't correct, the numbers will need to be something like -A A/2 A -A/2 where A is something like pi * radius i.e pi * 6378137 Your source might have this information stored somewhere, or a guess and a pray

Re: [gdal-dev] Abou R sf library and GDAL

2024-10-28 Thread Michael Sumner via gdal-dev
It's not RStudio or sf specific though, I think it's worth asking here because there are many many R packages using GDAL and they essentially share the same simple installation, after the GDAL version choice has been made. What really matters is your operating system and the method you have availab

Re: [gdal-dev] Reprojecting image with lat/lon bands - best practice?

2024-10-18 Thread Michael Sumner via gdal-dev
Nice one, thanks! (I had a fomenting PR to implement on the fly geolocation array config, so I'll use your hallucination for the design phase). 🤟 On Sat, 19 Oct 2024, 14:44 Even Rouault, wrote: > > Le 18/10/2024 à 23:32, Michael Sumner a écrit : > > I didn't know you could do that with -to!! Th

Re: [gdal-dev] Reprojecting image with lat/lon bands - best practice?

2024-10-18 Thread Michael Sumner via gdal-dev
I didn't know you could do that with -to!! That's awesome 🤟 On Sat, 19 Oct 2024, 05:01 Even Rouault via gdal-dev, < gdal-dev@lists.osgeo.org> wrote: > Conrad, > > Try something like: > > gdal_translate input.tif imagery.vrt -b 3 > > gdalwarp imagery.vrt imagery_warped.tif -geoloc -to X_DATASET=i

Re: [gdal-dev] Reprojecting image with lat/lon bands - best practice?

2024-10-18 Thread Michael Sumner via gdal-dev
Please provide a link to the file, or the full gdalinfo output with -stats 👌 On Fri, 18 Oct 2024, 23:16 Conrad Bielski, wrote: > Hi Mike, > thanks for the prompt. Looking into it, the data are DEM corrected > latitude/longitude. A GDAL info shows it as a regular raster band: Band 1 > Block=4865x

Re: [gdal-dev] Reprojecting image with lat/lon bands - best practice?

2024-10-18 Thread Michael Sumner via gdal-dev
Check if gdalinfo sees the longitude and latitude as geolocation arrays, then just warp to grid you want. If they aren't auto identified this can be arranged with VRT. Cheers, Mike On Fri, 18 Oct 2024, 21:55 Conrad Bielski via gdal-dev, < gdal-dev@lists.osgeo.org> wrote: > Hi Javier, > > it is

[gdal-dev] Zarr v2 and compression problem with small blocks

2024-10-14 Thread Michael Sumner via gdal-dev
When I use compression for ZARR_V2 with these small block sizes, I get an error: gdal_translate "/vsicurl/ https://github.com/cran/png/blob/master/inst/img/Rlogo.png?raw=true"; lz4.zarr -of ZARR -co FORMAT=ZARR_V2 -co BLOCKSIZE=4,25,26 -co COMPRESS=LZ4 -co INTERLEAVE=BAND Input file size is 100,

Re: [gdal-dev] 2D multiband to 3D MULTIDIM results in duplicated "x" dimension

2024-10-14 Thread Michael Sumner via gdal-dev
o/gdal/pull/11005 > > Otherwise just do "gdal_translate Rlogo.png Rlogo.zarr -of Zarr" . By > default the Zarr driver translates GDAL bands as the Z dimension, unless > you specify the SINGLE_ARRAY=NO creation option, in which case it will > create a Zarr array per band (so a di

[gdal-dev] 2D multiband to 3D MULTIDIM results in duplicated "x" dimension

2024-10-14 Thread Michael Sumner via gdal-dev
Possibly this is a fool's errand ... I want to be able to cast an RGBA file as a Zarr, so first to cast as multidim I tried the CreateCopy route with NetCDF and I see dimension "x" repeated, with "y" missing. (I'm using an external file as the existing test PNG files are square in shape). from os

Re: [gdal-dev] writing arrow geometry

2024-10-07 Thread Michael Sumner via gdal-dev
> Michael, > > > > > > my understanding of https://geoarrow.org/format.html#memory-layouts > is that what writes OGR is supposed to be fine since they mentionned types > like 'List[2]>>'. Perhaps I've missed something > or nanoarrow has st

Re: [gdal-dev] writing arrow geometry

2024-10-07 Thread Michael Sumner via gdal-dev
I realize I left out the INTERLEAVING, ie. ogr2ogr ~/fromgdal.arrow ogr/data/arrow/from_paleolimbot_geoarrow/polygon-default.ipc -lco GEOMETRY_ENCODING=GEOARROW_INTERLEAVED but still, I get these list>>}>> On Tue, Oct 8, 2024 at 12:19 AM Michael Sumner wrote: > When I investigate the schema

[gdal-dev] writing arrow geometry

2024-10-07 Thread Michael Sumner via gdal-dev
When I investigate the schema in one of the test files ogr/data/arrow/from_paleolimbot_geoarrow/polygon-default.ipc I see expected listGetArrowStream and I get the same output: }>>}>> If I write a new .arrow with GDAL ogr2ogr ~/fromgdal.arrow ogr/data/arrow/from_paleolimbot_geoarrow/polygon-d

Re: [gdal-dev] kerchunk

2024-07-24 Thread Michael Sumner via gdal-dev
gt; them. > > And obviously one strong assumption of Kerchunk is that the files > referenced by a Kerchunk index are immutable. If for some reason, tiles are > moved internally because of updates, chaos will arise due to (offset, size) > tuples being out of sync. > > Even &

Re: [gdal-dev] kerchunk

2024-07-23 Thread Michael Sumner via gdal-dev
On Wed, Jul 24, 2024 at 8:37 AM Michael Sumner wrote: > Hi, is there any effort or thought into something like Python's kerchunk > in GDAL? (my summary of kerchunk is below) > > https://github.com/fsspec/kerchunk > > I'll be exploring the python outputs in detail and looking for hooks into > wh

Re: [gdal-dev] kerchunk

2024-07-23 Thread Michael Sumner via gdal-dev
____________ > From: gdal-dev on behalf of Michael > Sumner via gdal-dev > Sent: Tuesday, July 23, 2024 17:37 > To: gdal-dev > Subject: [gdal-dev] kerchunk > > Hi, is there any effort or thought into something like Python's kerchunk > in GDAL? (my summary of kerch

[gdal-dev] kerchunk

2024-07-23 Thread Michael Sumner via gdal-dev
Hi, is there any effort or thought into something like Python's kerchunk in GDAL? (my summary of kerchunk is below) https://github.com/fsspec/kerchunk I'll be exploring the python outputs in detail and looking for hooks into where we might bring some of this tighter into GDAL. This would work

Re: [gdal-dev] can GDAL render two images as if over-plotted?

2024-07-08 Thread Michael Sumner via gdal-dev
Just an update, I have found out how to do this numerically.Will consider a PR for gdaldem if I can make it work at some point. Thanks, Mike On Tue, Jul 9, 2024 at 7:31 AM Michael Sumner wrote: > Is it possible to "layer" one image over another, the first at full > transparency and the se

[gdal-dev] can GDAL render two images as if over-plotted?

2024-07-08 Thread Michael Sumner via gdal-dev
Is it possible to "layer" one image over another, the first at full transparency and the second with partial transparency? I'm thinking of combining a hillshade and a (partially) transparent color-relief image, a visual enhancement technique. (I know how to do this downstream in R and Python or

Re: [gdal-dev] gdalwarp across the antimeridian with a source extent >180E

2024-06-03 Thread Michael Sumner via gdal-dev
27;t cover more > than 360 degrees of longitude. This value, when computed, is passed as a > hint OGRCoordinateTransformation so that it can post-correct longitudes to > apply a +/- 360 degree offset, to be in the range of the source dataset. > I've relaxed the sanity check to al

Re: [gdal-dev] gdalwarp across the antimeridian with a source extent >180E

2024-05-31 Thread Michael Sumner via gdal-dev
I forgot to mention that it needs earthdata credentials set up, basically your "Authorization: Bearer " in GDAL_HTTP_HEADERS or similar config. https://urs.earthdata.nasa.gov/documentation/for_users/user_token You can't download or stream these files without that set or logging in (the file used

[gdal-dev] gdalwarp across the antimeridian with a source extent >180E

2024-05-31 Thread Michael Sumner via gdal-dev
This data source has an odd georeferencing, it's a 36000x17999 raster in the extent -179.995 -89.995 180.005 89.995. vrt://NETCDF:/vsicurl/ https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2024010109-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc:analy

[gdal-dev] clear band level metadata

2024-05-29 Thread Michael Sumner via gdal-dev
Hello, I have this process to convert a URL to a MEM dataset. It's not the target data but it's representative, a netcdf with band-level and dataset-level metadata. We can clear the dataset level with COPY_SRC_MDD, but I can't see how to do that for the band level. We're hoping to keep this witho

Re: [gdal-dev] another problem HDF5

2024-05-01 Thread Michael Sumner via gdal-dev
lue would be good enough to handle that outlier. > > (Someone, super bored, should author a book about with horror stories with > netCDF and HDF georeferencing. There's a lot of material. Although likely > not to be a best seller) > > Even > Le 30/04/2024 à 09:45, M

Re: [gdal-dev] gdal2tiles for floating point images

2024-04-30 Thread Michael Sumner via gdal-dev
Ok so my naive edits are clearly not enough, they still get written as Byte so it's deeper in the target spec and spread across a few places I'm not ready to get across yet. Happy to pursue in the longer term though. Cheers, Mike On Tue, Apr 30, 2024 at 7:00 AM Michael Sumner wrote: > > > On

[gdal-dev] another problem HDF5

2024-04-30 Thread Michael Sumner via gdal-dev
This time, with: https://n5eil01u.ecs.nsidc.org/ATLAS/ATL20.004/2018.10.14/ATL20-01_20181001010615_00370101_004_01.h5 NETCDF gets the geotransform (from x_grid,y_grid which report in the metadata) but no crs: gdalinfo NETCDF:ATL20-02_20181001010615_00370101_004_01.h5 -sd 1 -nomd Driver: netCDF

Re: [gdal-dev] HDF5 and geolocation arrays

2024-04-29 Thread Michael Sumner via gdal-dev
t; ERROR 3: Cannot read 4029 bytes > > gdalinfo failed - unable to open '/vsicurl/ > https://login:pas...@n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5 > '. > > > > > > Thanks > > > > Joaquim > > > > *Fr

Re: [gdal-dev] gdal2tiles for floating point images

2024-04-29 Thread Michael Sumner via gdal-dev
On Fri, Apr 26, 2024 at 5:37 AM lefsky--- via gdal-dev < gdal-dev@lists.osgeo.org> wrote: > I'd like to have a version of gdal2tiles that handles image types other > than uint8. Is there a reason why it doesn't handle those types of images? > It appears I'd have to modify the output format from pn

Re: [gdal-dev] HDF5 and geolocation arrays

2024-04-29 Thread Michael Sumner via gdal-dev
0, 435.000) ( 42d14'27.21"E, 39d11'27.54"S) > Lower Right ( 395.000,-395.000) (135d 0' 0.00"E, 41d23'59.41"S) > Center ( 0.000, 20.000) ( 0d 0' 0.01"E, 88d 8'51.76"S) > Band 1 Bl

[gdal-dev] HDF5 and geolocation arrays

2024-04-29 Thread Michael Sumner via gdal-dev
This HDF5 (requires earthdata credentials your "Authorization: Bearer " in GDAL_HTTP_HEADERS, or equiv) presents without geolocation arrays. gdalinfo "/vsicurl/ https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5"; -sd 26 Driver: HDF5Image/HDF5 Dataset

Re: [gdal-dev] lost python api doc?

2024-04-28 Thread Michael Sumner via gdal-dev
.warp on gdal.org directly : > https://gdal.org/search.html?q=gdal.warp&check_keywords=yes&area=default > > that leads to https://gdal.org/api/python/utilities.html#osgeo.gdal.Warp > > Even > Le 29/04/2024 à 00:52, Michael Sumner via gdal-dev a écrit : > > I'm confu

[gdal-dev] lost python api doc?

2024-04-28 Thread Michael Sumner via gdal-dev
I'm confused about how to browse the python api docs, I used to just web search "osgeo.gdal Warp", and then scan down the first result page to find "Warp(" and I had the documentation I needed. Where is that now? (Why can't I find it, sorry - grepping the sources for "Warp(" only results in RFC 59

Re: [gdal-dev] Reading interpolated values on DSM

2024-04-24 Thread Michael Sumner via gdal-dev
Or a grouping function that returned the cell index for neighbours and weighting that are involved in whatever calculation summary is wanted. Maybe the warper could return this as a starting point rather than doing the "task at hand". ? On Wed, Apr 24, 2024 at 8:51 PM Even Rouault via gdal-dev

[gdal-dev] advice on python style - get file/dataset as bytes

2024-04-14 Thread Michael Sumner via gdal-dev
Hi, I'm getting some pushback on my code style. I just want the raw bytes in-memory of a file in Python, for a manageable tiny dataset. Is there anything especially wrong with the following? It's essentially the same as the answer here: https://lists.osgeo.org/pipermail/gdal-dev/2016-August/045

Re: [gdal-dev] discussion on xarray and the missing abstract-grid specification

2024-04-03 Thread Michael Sumner via gdal-dev
> the 1D coordinate array with a very small relative epsilon, like 1e-8, is > sufficient. > > Even > > > Le 03/04/2024 à 22:56, Michael Sumner via gdal-dev a écrit : > > Here's an (ahem) extremely important discussion on the prospects for > xarray to extend from t

[gdal-dev] discussion on xarray and the missing abstract-grid specification

2024-04-03 Thread Michael Sumner via gdal-dev
Here's an (ahem) extremely important discussion on the prospects for xarray to extend from the coordinates-only model (like that of netcdf) for geo reference: https://discourse.pangeo.io/t/example-which-highlights-the-limitations-of-netcdf-style-coordinates-for-large-geospatial-rasters/ I'm heart

Re: [gdal-dev] STACIT and limit

2024-04-03 Thread Michael Sumner via gdal-dev
Oh, my bad. It's documented to split into SDS for multiple CRS. Ouch that's not how I thought it was working, but it makes sense. Thanks, Mike On Thu, Apr 4, 2024 at 1:26 AM Michael Sumner wrote: > this works for me, there are ten items in the filelist: > > gdalinfo "STACIT:\" > https://eart

Re: [gdal-dev] HDF5 and identified fields / primary dimension

2024-04-03 Thread Michael Sumner via gdal-dev
> For that particular file, I see that the "feature_id" variable > (corresponding to the "feature_id" dimension) has a cf_role = > "timeseries_id" attribute, and that the global metadata has a > featureType = "timeSeries" attribute. So given > > https://cfconventions.org/Data/cf-conventions/cf-conv

[gdal-dev] STACIT and limit

2024-04-03 Thread Michael Sumner via gdal-dev
this works for me, there are ten items in the filelist: gdalinfo "STACIT:\" https://earth-search.aws.element84.com/v1/search?collections=sentinel-2-c1-l2a&bbox=0,0,10,10&datetime=2023-01-01T00:00:00Z/2023-12-31T23:59:59Z\":asset=visual"; -oo MAX_ITEMS=10 and with 20 it's also fine, filelist of 20

Re: [gdal-dev] HDF5 and identified fields / primary dimension

2024-04-01 Thread Michael Sumner via gdal-dev
well actually, I think what I'm asking for is the intended behaviour, but there's an error. Is it meant to detect sets of variables on 1D dimensions and present them as layers? That's what would make sense to me. Still exploring. Cheers, Mike On Tue, Apr 2, 2024 at 5:36 AM Michael Sumner wro

[gdal-dev] HDF5 and identified fields / primary dimension

2024-04-01 Thread Michael Sumner via gdal-dev
This source has an array on 'feature_id' with 2729077 values, with various fields elevation, longitude, latitude, qBtmVertRunoff, qBucket, etc '/vsis3/noaa-nwm-retro-v2.0-pds/full_physics/2017/20170401.CHRTOUT_DOMAIN1.comp' It is accessible via the mdim api. Structurally it is basically a t

Re: [gdal-dev] palette file format doc ?

2024-03-20 Thread Michael Sumner via gdal-dev
> > > > > And, can index be *value* in any contexts? > > If you use a raster with a signed data type, that could be negative > values (assuming I understand your question) > > ah I see, arbitrary integer values map to a colour 0:(n-1) colours, match the ordered n values in the raster - that is "va

[gdal-dev] palette file format doc ?

2024-03-20 Thread Michael Sumner via gdal-dev
Is the palette_file .txt format documented? https://gdal.org/programs/gdalattachpct.html It's mentioned in a few utilities, and created by tests but I couldn't find an existing example or a description (I guessed, incorrectly at first, leaving out the index column). I take it that it is 0-255 ra

Re: [gdal-dev] overview sizes for COG

2024-03-19 Thread Michael Sumner via gdal-dev
Excellent! Thanks for the answers. When I've explored a bit more I might implement the overview sizes just so we can match downstream tools (the current motivation is like-for-like performance comparison, I haven't looked there but I think odc goes very low level to eke out speed). Cheers, Mike

[gdal-dev] overview sizes for COG

2024-03-19 Thread Michael Sumner via gdal-dev
Hi, can we specify overview sizes exactly? I have this odd grid that is 36000x17999, and I get consequently yucky overview sizes: gdal_create -outsize 36000 17999 -ot Int8 -co SPARSE_OK=YES -a_srs EPSG:4326 -a_ullr 0 17999 0 36000 weird.tif gdal_translate weird.tif cog.tif -of COG gdalinfo cog.ti

[gdal-dev] netcdf url : should lack of range support be the error?

2024-02-14 Thread Michael Sumner via gdal-dev
on this URL I get an error, and as far as i understand - the server doesn't support range downloading, the file otherwise works fine. gdalinfo failed - unable to open '/vsicurl/ https://erddap.emodnet.eu/erddap/files/biology_6640_benthos_NorthSea_e4af_0f0e_6a73/04_2021_6640_diva_benthos_erddap.nc

Re: [gdal-dev] core dump on dir info

2024-02-06 Thread Michael Sumner via gdal-dev
I don't understand how jammy is "old" when the full build is itself using "BASE_IMAGE=ubuntu:22.04" But, I'm out of my depth in these emails and trying to learn, thanks! Cheers, Mike On Wed, 7 Feb 2024, 00:46 Javier Jimenez Shaw, wrote: > Could you set up your VMs to include those SSE instr

Re: [gdal-dev] core dump on dir info

2024-02-05 Thread Michael Sumner via gdal-dev
;>>> ==704== Memcheck, a memory error detector >>>> ==704== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. >>>> ==704== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright >>>> info >>>> ==704== Command: ogrinfo /

Re: [gdal-dev] core dump on dir info

2024-02-04 Thread Michael Sumner via gdal-dev
. >>> 1: geo_melbourne (Polygon) >>> ==704== >>> ==704== HEAP SUMMARY: >>> ==704== in use at exit: 25,486 bytes in 216 blocks >>> ==704== total heap usage: 15,761 allocs, 15,545 frees, 2,390,169 bytes >>> allocated >>> ==704

Re: [gdal-dev] core dump on dir info

2024-02-04 Thread Michael Sumner via gdal-dev
t;> ==704== suppressed: 0 bytes in 0 blocks >> ==704== Rerun with --leak-check=full to see details of leaked memory >> ==704== >> ==704== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) >> >> >> ogrinfo /tmp/newdir >> Illegal instruc

Re: [gdal-dev] core dump on dir info

2024-02-03 Thread Michael Sumner via gdal-dev
ore dumped) > > Cheers, Mike > > > > > On Sat, Feb 3, 2024 at 12:46 PM Even Rouault > wrote: > >> Michael, >> >> I'm wondering if there not might be something wrong with your build or >> runtime environment. Or there's something subtle, because

Re: [gdal-dev] core dump on dir info

2024-02-02 Thread Michael Sumner via gdal-dev
--args ogrinfo > /tmp/newdir/" (type "run") to get more useful information > > Even > Le 03/02/2024 à 02:35, Michael Sumner via gdal-dev a écrit : > > I'm getting Illegal instruction / core dumped on ogrinfo of a directory: > > ogr2ogr /tmp/newdir > ht

[gdal-dev] core dump on dir info

2024-02-02 Thread Michael Sumner via gdal-dev
I'm getting Illegal instruction / core dumped on ogrinfo of a directory: ogr2ogr /tmp/newdir https://github.com/SymbolixAU/geojsonsf/raw/master/inst/examples/geo_melbourne.geojson -f "ESRI Shapefile" ogrinfo /tmp/newdir/ Illegal instruction (core dumped) I've worked back through some docker imag

Re: [gdal-dev] Virtual Raster Tile Index (VRTTI) driver, and associated gdaltindex improvements

2024-01-30 Thread Michael Sumner via gdal-dev
just to follow up, I got it all working in latest GDAL: script at https://github.com/mdsumner/cog-example/blob/main/gti/cop90.py creates dsn /vsicurl/https://github.com/mdsumner/cog-example/raw/main/gti/cop90.gti.fgb which works nicely, thanks! Cheers, Mike On Tue, Jan 30, 2024 at 10:57 PM

Re: [gdal-dev] Virtual Raster Tile Index (VRTTI) driver, and associated gdaltindex improvements

2024-01-30 Thread Michael Sumner via gdal-dev
ah thanks, all very helpful - no it's not done with master (for various reasons), I'll follow up with details if relevant, mostly I was just excited to get a useable workflow for the entire process. Cheers, Mike On Tue, 30 Jan 2024, 22:46 Even Rouault, wrote: > Michael, > > You need to attach t

Re: [gdal-dev] Virtual Raster Tile Index (VRTTI) driver, and associated gdaltindex improvements

2024-01-30 Thread Michael Sumner via gdal-dev
awesome, thanks Even I'm having fun with this one. For anyone interested I created Python to parse the OpenTopography COP90 VRT (I have to wget it locally as I don't know how to hit the URL for the xml yet). https://github.com/mdsumner/cog-example/blob/1ca74f3baffe69830180031ddabcbbc569816150/gti

Re: [gdal-dev] Layerstack multiple images, of different extents and variable number of bands

2024-01-24 Thread Michael Sumner via gdal-dev
I would start with gdalwarp out.tif set -ts to something small at first to get a visual like -ts 1024 0 (the y zero means the aspect ratio is figured out sanely from the x size - with no arguments you get best-preserving grid from all resolved inputs - set some or 3 of -t_srs -tr -te -ts for

Re: [gdal-dev] help please with close/reopen and GDALSubdatasetInfo

2023-12-03 Thread Michael Sumner via gdal-dev
L_OF_RASTER, nullptr, nullptr, nullptr); > CPLFree(pszSubdatasetSource); > break; >} >else { > CPLFree(pszSubdatasetSource); >} > } >} > >poSrcDS->ReleaseRef(); >return 1; > } > > Even > >

[gdal-dev] help please with close/reopen and GDALSubdatasetInfo

2023-12-02 Thread Michael Sumner via gdal-dev
May I please ask for assistance with this code? I'm trying to close a dataset with subdatasets and then reopen if the requested subdataset is found. It segfaults for all but the last subdataset name ... and I'm at a loss for what I'm doing wrong. Thank you. Code below, and this gist documents i

[gdal-dev] slow translate with OVERVIEW_LEVEL=NONE when no overviews exist

2023-11-24 Thread Michael Sumner via gdal-dev
When I translate this GeoTIFF to 10% original size, it's very very slow if OVERVIEW_LEVEL=NONE is set. The GeoTIFF has no overviews. export dsn="/vsicurl/ https://github.com/mdsumner/cog-example/raw/main/cog/sentinel-image.tif"; ## takes *forever* gdal_translate $dsn out.tif -outsize 219 226 -oo

[gdal-dev] format for topology?

2023-10-09 Thread Michael Sumner via gdal-dev
Are there any formats that record "coverage" topology? What I'm worried about is when shapes are encoded as blob geometry with initially identical coordinates at shared vertices, is there any process that can validate or record that particular coords should have the same values even after reprojec

Re: [gdal-dev] Using ogr2ogr with limited memory

2023-09-29 Thread Michael Sumner via gdal-dev
what's the task? what about batching the geometry and or fields? can you run on just the first feature, does that work? how many features, how big is the task? On Wed, 27 Sept 2023, 13:16 Scott via gdal-dev, wrote: > Any tips for using ogr2ogr to use only a specified amount of RAM? I'm > not ha

[gdal-dev] resampling algs warp vs translate

2023-09-21 Thread Michael Sumner via gdal-dev
Can we aspire to translate having the same set of resampling algorithms as the warper? I see the warper adds min, max, mod, q1, q3, sum I especially wanted sum for OVERVIEW_RESAMPLING in COG, and I can see where it's done and ... can maybe see my way through that ... but the 600 lines of code i