Re: [gdal-dev] DuckDB queries in GDAL/OGR master branch

2024-12-27 Thread Scott via gdal-dev
Yes, s3 (httpfs) and of course local (filesystem) both have read/write capabilities. httpfs is the reason duckdb works over network. http(s) is the most ubiquitous internet protocol/data storage. geoparquet files on a web server http(s) *will not work*. Drop FGB on any http* server and it just

Re: [gdal-dev] DuckDB queries in GDAL/OGR master branch

2024-12-27 Thread Michael Smith via gdal-dev
I've used it with geoparquet files from s3, local geopackage files etc. Duckdb works well with geoparquet, for the other formats its using GDAL to read. You certainly can use flatgeobuf. Yep, for just bbox queries, no need for alternative sql methods. This is more for doing more data analytics

Re: [gdal-dev] DuckDB queries in GDAL/OGR master branch

2024-12-27 Thread Scott via gdal-dev
Thanks for the example Michael! Will this work with geoparquet files on an out of the box apache web server? Will I need to replace my simple apache server(s) with httpfs/hdfs/hadoop or forced to use public, proprietary cloud storage? If I *only* need to filter by bbox, FlatGeoBuf (FGB) for

[gdal-dev] DuckDB queries in GDAL/OGR master branch

2024-12-27 Thread Michael Smith via gdal-dev
With some recent commits to GDAL master branch, you can now do some amazing DuckDB queries in GDAL! For example, h3 level 6 indexing for POIs in NH from Overture Maps ogr2ogr -f parquet POI_NH_H3.parquet -nlt POINT -a_srs epsg:4326 -oo ADBC_DRIVER=libduckdb -oo PRELUDE_STATEMENTS="LOAD S