Hi,
Somewhat related to some of the discussions in that thread, to be able
to read "uncommon" (to remain polite) GeoTIFF products generated by the
Sentinel Application Toolbox (https://github.com/OSGeo/gdal/pull/10867),
I found that using libtiff, libgeotiff or the GDAL GTiff driver wasn't
going to be practical, so I wrote my own TIFF library, dubbed
"libertiff": https://github.com/libertiff/libertiff
C++11, no variable argument brain damage like in libtiff, header only,
read-only, no codec support for now (no actual pixel reading capability
in fact, just telling you the offset&size of tiles&strips. yes, that's a
big limitation :-)), thread-safe by design.
Fits in 1,500 lines of code or so:
https://github.com/libertiff/libertiff/blob/master/libertiff.hpp. Could
maybe serve as a foundation for a potential future thread-safe read-only
GDAL GeoTIFF driver.
Even
Le 06/09/2024 à 21:57, Frank Warmerdam a écrit :
Even,
I'm actually surprised that it is considered impractical to make
some/most drivers thread safe (read-only) natively and I thought key
ones already were mostly there in the past though I'm not so confident
in my memory. Looking through the GTiff code I find myself not able
to work out where locking is done and how we are managing the
directory loading/switching as block reads are done.
I would certainly like to work towards read-only thread safety for
some drivers with:
- a shared block cache for all threads reading from a dataset
- no duplication of file handles per-thread
- a way of checking that a dataset is considered to be thread safe.
I'm curious if you can talk a bit more about the challenges of
accomplishing the above. I had been hoping to take advantage of my
(assumption) that this already mostly existed in GDAL for our new high
performance environment at Planet so it is of more than academic
interest to me. Also, doesn't gdalwarp's multithreading support
already depend on read-only thread safety for source files?
The above said, I can also see some utility in RFC 101 and have no
objection to it.
Best regards,
Frank
On Fri, Sep 6, 2024 at 12:45 PM Kurt Schwehr via gdal-dev
<gdal-dev@lists.osgeo.org> wrote:
My take would be to not offer the force option unless there is
someone who states they really need it.
On Fri, Sep 6, 2024 at 6:52 AM Even Rouault via gdal-dev
<gdal-dev@lists.osgeo.org> wrote:
Hi,
No other reactions? In particular w.r.t. the open question
whether it is
worth to offer a "force" mode in the situations where a
dataset cannot
be re-opened (or for a band whose belonging to a dataset
cannot be
determined) where the thread-safe wrapper would just put a
mutex around
the calls to that dataset/band (so obviously this wouldn't
scale with
the number of threads). I'm probably leaning on not offering
this option
for now, if there's no obvious need it is useful. It can
always been
added afterwards.
Even
Le 29/08/2024 à 14:26, Even Rouault via gdal-dev a écrit :
> Hi,
>
> I've worked on a new RFC 101: Raster dataset read-only
thread-safety:
> https://github.com/OSGeo/gdal/pull/10676
>
> Summary: This RFC enables users to get instances of
GDALDataset (and
> their related objects such as GDALRasterBand) that are
thread-safe for
> read-only raster operations, that is such instances can be
safely used
> from multiple threads without locking.
>
> It is backed by a preliminary implementation (fairly
complete, apart
> from writing new tests...). There's at least one open
question raised
> in the RFC.
>
> PS: you can see one of the benefits of the migration of the
doc to
> ReadTheDocs hosting, with the preview of the updated
documentation in
> pull requests. So this is a way to mention that the move to RtD
> hosting has now been completed, with the default version
("latest")
> reflecting the state of the master branch, as well as the
version for
> the release/3.9 branch. Thanks to Dan Baston (and other people
> involved) for this work! There might be a few adjustments
needed, so
> if you notice something unusual w.r.t. to the previous
state, please
> file a ticket.
>
> Even
>
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | USA: +1 650-701-7823
and watch the world go round - Rush | CAN: +1 343-550-9984
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
