Hi, > 1) There are a lot of knobs that can be used to tune the thing that are not > documented. For example CPL_VSIL_CURL_USE_CACHE. Is it on purpose?
Yes, the disk cache is an experiment that isn't used anywhere (from what I know) and likely not in a finished state as you noticed in your below points which are all valid and should be addressed if someone wanted to make it production ready. > > 6) If VSI_CACHE is enabled the data is cached twice in memory (papsRegions > and VSICachedFile). Is it wanted? The scope of the caches are not the same. papsRegions is a global cache shared by all / vsicurl/ handles, and persistant (in memory) on their closing (so that if the same filename is closed and re-opened in sequence, already read parts can be reused), whereas VSICachedFile is associated with a single file handle. I guess there could be some optimizations to avoid those duplications, but that could complicate substantially the code which is already non trivial. > > 7) If the file's content is modified, it's the total mess. We'll end up > having portions of the file having the old data while the rest has the new > data. I'm quite sure the GeoTiff we end up with won't be very valid. Indeed. But the mess would also happen with no caching mechanism if a file is modified while being read. Even for a local file, GDAL using glibc FILE buffering API, so even if you modify some portions of a GeoTIFF that haven't been read yet, but you already read closing regions, there's a chance, you'll read old data in part. > > 8) In the case discussed in 7), CPL_VSIL_CURL_NON_CACHED will just purge > the data from 1 the 3 caches: papsRegions. The vsil_cache and the disk will > still cache the content. CPL_VSIL_CURL_NON_CACHED avoids the content of a file to be preserved in the papsRegion cache when a file handle is closed and re-opened. And VSICachedFile is only valid during the lifetime of the file handle. So I don't think there's an issue there. Perhaps the naming CPL_VSIL_CURL_NON_CACHED is a bit misleading: there's always some cache (otherwise /vsicurl performance would be just too horrible), it is just that it doesn't survive file closing. Even -- Spatialys - Geospatial professional services http://www.spatialys.com
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
