Hi Darren
You can do this by setting `GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR` or by
narrowing the allowed extension with "CPL_VSIL_CURL_ALLOWED_EXTENSIONS=.tif"
Kyle is working on some docs that explains those env in TiTiler:
-
https://github.com/developmentseed/titiler/blob/0a5288de30845865256
In https://github.com/mapbox/rasterio/issues/2119#issuecomment-790024225
I've noted how s3-COG reads can use retries to handle s3 rate throttling
(503) by enabling the `GDAL_HTTP_MAX_RETRY`. Our dataset on s3 only
contains *.tif files and we need to prevent GDAL from issuing any GET/HEAD
requests
