thanks for the prompt response. It would have been nice to use the GDAL API in a multi-process (parallelised) world if feasible. We are increasingly seeing the need to move to multi-core (parallel) access so we can spread the processing load across multiple CPU's, but we do like to keep our data in a single, cohesive raster file (often with billions of pixels) so we can avoid stitching errors at the seams. The use of pyramids means the visualisation of huge rasters is not an issue but single write access is starting to be a problem. We are thinking of a 'master' process running which can server out tiles to different processes and keep track of them to avoid write collision. Almost a Tile Map Service with write capabilities. would you think this is feasible with GDAL as the backend service?
-----Original Message----- From: fwarmer...@gmail.com [mailto:fwarmer...@gmail.com] On Behalf Of Frank Warmerdam Sent: Friday, 11 January 2013 9:33 AM To: Kennedy, Paul Cc: gdal dev Subject: Re: [gdal-dev] does gdal support multiple simultaneous writers to raster Paul, Generally speaking - no - GDAL does not now support multi-threaded writing. Part of this is due to the way stuff gets write-flushed out of the block cache and partly it is just that none of GDAL supports multiple threading on a single dataset. This statement is based on the assumption it is all in one process. If you want to run different processes with the output to a pre-existing "fixed location" format like uncompressed GeoTIFF with tiles that is all preinitialized to some real value then you could certainly use one output file for many writers as long as case is taken to not have multiple processes write to the same tile. But GDAL isn't doing anything to make this easy. Best regards, Frank On Thu, Jan 10, 2013 at 4:12 PM, Kennedy, Paul <p.kenn...@fugro.com.au> wrote: > I am looking to parallelize a long duration gridding process, and was > wondering if GDAL can support multiple simultaneous writers from processess? > > > > I am not sure if any of the underlying raster formats support this? > > > > thanks in advance. > > > > regards > > pk > > > > > > > _______________________________________________ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---------------------------------------+-------------------------------- ---------------------------------------+------ I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush | Geospatial Software Developer _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
