Hi,
Why not reading small chunks (i.e. 256 x 256) with RasterIO
<http://www.gdal.org/classGDALDataset.html#ae077c53268d2272eebed10b891a05743>
from big raster? We'll get less memory usage.
The program can count the percentage using the size of GDALDataset and
size of chunks and cancel on any chunk.
By the way, can GDALDataset RasterIO different chunks from separate threads?
Best regards,
Dmitry
12.12.2013 19:28, Even Rouault ?????:
Selon Martin Dobias <wonder...@gmail.com>:
Hi Even
thanks for your thoughts.
On Thu, Dec 12, 2013 at 8:30 PM, Even Rouault
<even.roua...@mines-paris.org> wrote:
- rasters - currently we do not have API to cancel requests for raster
blocks. This means that currently we need to wait until the raster
block is fully read even when we cancel the rendering job. GDAL has
some support for asynchronous requests - anyone has some experience
with it?
Indeed there's an API in GDAL for asynchronous requests :
http://trac.osgeo.org/gdal/wiki/rfc24_progressive_data_support
Note that it has a "real" implementation only in the JPIPKAK driver
(driver that implement the JPIP protocol - JPEG2000 trough the network -
with the Kakadu library). For other drivers that don't have a dedicated
implementation, a default implementation will just turn the async requests
as a sync request (more precisely GetNextUpdatedRegion() will just do
a RasterIO(), other methods mentionned in the RFC are no-op).
Real cancellation / async request is generally hard to implement since the
low level libraries used by drivers must support it. The analysis must be
done on a per-driver basis (determine which drivers are important for you,
which ones are slow, see if it can be remedied in the general synchronous
case, otherwise imagine how async request could be implemented, ...)
We could imagine that the default GetNextUpdatedRegion(), instead of
issuing a single RasterIO() could split it up into several RasterIO()
if the request window crosses several tiles (in the case of a tiled
dataset) to allow cancellation between the calls to GetNextUpdatedRegion().
But that's no always a beneficial strategy. Drivers can use the fact that
a RasterIO() is done on a "big" window to optimize things a bit, w.r.t
issuing several RasterIO() on smaller windows. In GDAL 1.10, I've for
example
implemented multi-threaded block decoding in the OpenJPEG driver : if the
window of a RasterIO() call spans over several tiles and you have several
cores, then each tile will be decoded in parallel.
Regarding the current API of asynchronous requests, I feel there's perhaps
a lack of a way of signaling that data is ready. GetNextUpdatedRegion()
works on a pulling startegy. Perhaps a callback mechanism could be usefull.
That topic needs some thinking...
So obviously adding async API support for other common formats would
take a lot of time... I was secretly hoping that the remark in rfc24
mentioning only jpipkak driver was just a temporary state of affairs
:-)
However I realize that actually I do not really need to have truly
asynchronous requests. The requests from QGIS are being made from
within a worker thread, so it is not a problem if GDAL block that
thread. The only thing I need is to be able to tell GDAL to stop
reading data as soon as possible once the rendering gets cancelled.
API-wise, I would like to have a method in GDALDataset, using which I
could register my custom function returning cancellation status of the
request (just true/false). The drivers would be responsible for
calling that function whenever appropriate during RasterIO to check
whether the request has been cancelled or not. It would be probably
much less hassle to implement this functionality in drivers - although
I understand that still many drivers using custom libraries won't be
able to provide that functionality.
Yes, that's a reasonable possibility. The callback could also take
as parameter a percentage of completion. That could actually be a
GDALProgressFunc (
http://trac.osgeo.org/gdal/browser/trunk/gdal/port/cpl_progress.h )
Regards
Martin
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
