Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-13 Thread Even Rouault via gdal-dev
Le 13/09/2024 à 19:40, Javier Jimenez Shaw a écrit : Thanks Even regarding GDAL_SKIP=TileDB as environment variable, it still shows the message [Fri Sep 13 19:37:03 2024].8370, 10.5869: GDAL: On-demand registering /home/jshaw/anaconda3/envs/qgis_stable/lib/gdalplugins/gdal_TileDB.so using GD

Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-13 Thread Javier Jimenez Shaw via gdal-dev
Thanks Even regarding GDAL_SKIP=TileDB as environment variable, it still shows the message [Fri Sep 13 19:37:03 2024].8370, 10.5869: GDAL: On-demand registering /home/jshaw/anaconda3/envs/qgis_stable/lib/gdalplugins/gdal_TileDB.so using GDALRegister_TileDB. On Fri, 13 Sept 2024 at 19:27, Even Ro

Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-13 Thread Even Rouault via gdal-dev
Le 13/09/2024 à 19:16, Javier Jimenez Shaw via gdal-dev a écrit : This is getting even weirder. I tried removing gdal_TileDB.so before my first email, and it didn't work. However, now it works... but only if I opened the file before. If I wait a couple of minutes, then it still waits 5 or 6 sec

Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-13 Thread Javier Jimenez Shaw via gdal-dev
This is getting even weirder. I tried removing gdal_TileDB.so before my first email, and it didn't work. However, now it works... but only if I opened the file before. If I wait a couple of minutes, then it still waits 5 or 6 seconds. Is there any cache somewhere? (I am closing QGIS, so it should n

Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-13 Thread thomas bonfort via gdal-dev
I also see that you have a tiledb driver registration happening only on your conda version, exactly at the time of your lag. you should also try to remove/rename your gdal_tiledb.so plugin file to check the issue isn't coming from there. Le ven. 13 sept. 2024, 18:22, thomas bonfort a écrit : > J

Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-13 Thread thomas bonfort via gdal-dev
Javier, 3.9 is probably doing parallel reads, which will cause a noticeable lag in requests on a slow connection (the overall processing time should not be slower anyhow). There's a gtiff open option documented that can disable this behavior, that you can try to use to see if it changes your timing

Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-13 Thread Javier Jimenez Shaw via gdal-dev
Thanks David for the suggestion. Very interesting data. Unfortunately the issue I have is not reproduced with that file. So it may be due to the authentication (your suggestion does not need authentication). Some context: I am opening a COG file from S3 in QGIS, setting the url in the command lin

Re: [gdal-dev] [EXTERNAL] s3 request example

2024-09-09 Thread Mayer, David P via gdal-dev
Would ~200 GB be large enough? Here is a mosaic of a section of Mars at 6 meters/pixel in a public S3 bucket: s3://asc-pds-services/mosaic/Mars/Mars_MRO_CTX_Equi_Mosaics_Robbins/6mpp/MC16_6mpp.16bit.tif Cheers, David --- David Mayer Geographer Astrogeology Science Center US Geological Su