Re: [gdal-dev] Python Bindings and Closed Datasets

2019-11-01 Thread Even Rouault
Patrick, with your updated script, the good & bad news is that I can reproduce it quite reliably, including with GDAL master on a debug build. I strongly suspect a subtle race issue in the global GDAL block cache or in the way the GTiff driver interacts with it, rather than something Python spe

Re: [gdal-dev] Python Bindings and Closed Datasets

2019-10-31 Thread Patrick Young
After some more investigation, I've managed to put together the script below that manifests the issue on my machine (8 core, Ubuntu 19.04, GDAL 2.4.0, Python 3.7). Where I first observed this was in using 8 band WV2 imagery, so that's what the complicated input image is modeled after. Here's an ex

Re: [gdal-dev] Python Bindings and Closed Datasets

2019-10-30 Thread Even Rouault
I've, unsuccessfully, tried to reproduce your issue with the following script: - from osgeo import gdal import concurrent.futures def worker(in_f, out_f): gdal.Unlink(out_f) gdal.Warp(out_f, in_f, options = '-co COMPRESS=DEFLATE -co TILED=YES -ts 2048 2048') jobs = [] for i in ran

Re: [gdal-dev] Python Bindings and Closed Datasets

2019-10-30 Thread Even Rouault
On mercredi 30 octobre 2019 12:05:01 CET Patrick Young wrote: > Hi all, > > I've been experiencing some behavior using the GDAL python bindings where I > am occasionally seeing what appears to be random blocks of the tiff being > unwritten in geotiffs I've pushed to S3. a small block(s) in one of

[gdal-dev] Python Bindings and Closed Datasets

2019-10-30 Thread Patrick Young
Hi all, I've been experiencing some behavior using the GDAL python bindings where I am occasionally seeing what appears to be random blocks of the tiff being unwritten in geotiffs I've pushed to S3. a small block(s) in one of the bands will be all zeros while everywhere else is good. My setup is