Hi Even, thanks for quick response and exhaustive information.
On Fri, Jul 26, 2013 at 2:37 PM, Even Rouault <even.roua...@mines-paris.org> wrote: > Selon Radim Blazek <radim.bla...@gmail.com>: > >> GDAL 1.9.2 fails to read (large?) overview files if >> VSI_CACHE=TRUE >> VSI_CACHE_SIZE=1000000 >> The problem is described here: http://hub.qgis.org/issues/8356. >> I just guess, that overview file keeps overview headers scattered over >> the file and VSI fails to set offset, 17891712530 in this case, over >> some limit. >> >> Could it be some hardcoded limit in VSI subsystem? Or just int used >> for offset instead of long? Is it easy to fix? Should I create a new >> ticket? > > Hi Radim, > > yes, you likely spotted a bug in the VSI Cache mechanism. Please file a GDAL > ticket. > > I've just done some quick code inspection in port/cpl_vsil_cache.cpp and I > suspect that the error comes from the signature of > > int VSICachedFile::LoadBlocks( size_t nStartBlock, size_t nBlockCount, > void *pBuffer, size_t nBufferSize ); > > and a few other uses of size_t in VSICachedFile::Read(). > > The size_t nStartBlock should likely be a vsi_l_offset. I guess the issue is > met > on a 32bit GDAL build ? On 64 bit, the error must not be hit as size_t will be > 64 bit. > > >> >> Until it gets fixed in GDAL, how can we easily enable the VSI cache >> for OGR and disable it for GDAL? > > Well, you could try that at each place where you GDALOpen(): > > /* yes strdup it since SetThreadLocalConfigOption() might invalidate the value > returned by CPLGetConfigOption() */ > char* pszOldVal = CPLStrdup(CPLGetConfigOption("VSI_CACHE", "FALSE")); > CPLSetThreadLocalConfigOption("VSI_CACHE", "FALSE"); > GDALDataset hDS = GDALOpen(xxxx); > CPLSetThreadLocalConfigOption("VSI_CACHE", pszOldVal); > CPLFree(pszOldVal); > > I'd suggest doing a wrapper function for the above, in case GDALOpen() is used > several times in QGIS code base... > > However.... in some cases, the opening of GDAL datasets is differed. This is > the > case for a VRT. If you open a VRT, GDAL opens the .vrt file, but doesn't open > immediately the tiles that compose it. So the above might not work reliably. > Or > you might need to wrap GDALRasterIO() similarly, and potentially other GDAL > API > that can indirectly lead to a GDALOpen() (statistics, histograms, ...) But the problem only happens with OVR file which is too big, the tiles are OK. I believe that OVR is opened together with VRT, (GDALOpen() is called form GDALOpen()?) so it should work just with GDALOpen() wrapper? Radim > Best regards, > > Even > > _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
