Some rather counterintuitive gdalwarp behavior: the bigger
dfWarpMemoryLimit (-wm setting) the more cpu time will be wasted on
warping not existing pixels. Why? Warping begins with the destination
window size of entire output image size. If this size is larger than
dfWarpMemoryLimit it is split in half along the longest axis and any
half that doesn't contain the currently processed source file is
discarded. With large dfWarpMemoryLimit this subdivision process will
stop early with still large portions of out of source image pixels.
Hermann Peifer wrote:
Frank Warmerdam wrote
Jukka Rahkonen wrote:
Conclusion: Too slow to be useful with my settings.
Question 1: Should it work this way?
Question 2: If yes, is it worth trying to find the limits when it
goes too slow
and perhaps file a ticket?
I hate to bring this up so late in the discussion, but have you tried
-wo SKIP_NOSOURCE=YES? When mosaicing small images into large images it
prevents processing of chunks for which there is no source data.
Otherwise
the whole output image is generated for each input image.
There is an open ticket on this issue to apply this option by default.
BTW, in combination with SKIP_NOSOURCE it can be helpful to keep the -wm
option to a modest size to the tiles aren't too large. A 200MB should be
sufficient - especially if the input and output image are tiled.
Just to repeat my earlier observations: over the last days, I tried with
all kinds of -wm and GDAL_CACHEMAX settings, and also added the options
TILED=YES and SKIP_NOSOURCE=YES. I haven't documented the performance
changes systematically, but my overall impression was that the changes
were rather small. I also tried -multi on my dual processor machine, but
I ended up with a fatal error.
What really helped was to follow Felix' approach, who wrote in the first
mail of this thread:
Now, a second script, pasting first "slices" of 5*30° together,
and than merging all the slices...
This way, one can reduce the processing time to some 10-20% compared to
the "all in one run approach". Concerning the size of the intermediate
slices: the smaller they are, the faster it goes. The final step of
gdalbuilding a vrt from the imtermediate slices and gdal_translating it
to the final output.tif is also very fast.
In order to avoid NODATA gaps in the final output.tif: It seems to be
safer if the intermediate slices are slightly overlapping, rather being
than just adjacent.
Hermann
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
