Adam Nowacki wrote:
Some rather counterintuitive gdalwarp behavior: the bigger
dfWarpMemoryLimit (-wm setting) the more cpu time will be wasted on
warping not existing pixels. Why? Warping begins with the destination
window size of entire output image size. If this size is larger than
dfWarpMemoryLimit it is split in half along the longest axis and any
half that doesn't contain the currently processed source file is
discarded. With large dfWarpMemoryLimit this subdivision process will
stop early with still large portions of out of source image pixels.
In other words, the counter-intuitive recommendation would be: The smaller the
-wm setting, the better?
Or say: the -wm value should better be just a bit bigger than the source image
size?
Hermann
Hermann Peifer wrote:
Frank Warmerdam wrote
Jukka Rahkonen wrote:
Conclusion: Too slow to be useful with my settings.
Question 1: Should it work this way?
Question 2: If yes, is it worth trying to find the limits when it
goes too slow
and perhaps file a ticket?
I hate to bring this up so late in the discussion, but have you tried
-wo SKIP_NOSOURCE=YES? When mosaicing small images into large images it
prevents processing of chunks for which there is no source data.
Otherwise
the whole output image is generated for each input image.
There is an open ticket on this issue to apply this option by default.
BTW, in combination with SKIP_NOSOURCE it can be helpful to keep the -wm
option to a modest size to the tiles aren't too large. A 200MB
should be
sufficient - especially if the input and output image are tiled.
Just to repeat my earlier observations: over the last days, I tried
with all kinds of -wm and GDAL_CACHEMAX settings, and also added the
options TILED=YES and SKIP_NOSOURCE=YES. I haven't documented the
performance changes systematically, but my overall impression was that
the changes were rather small. I also tried -multi on my dual
processor machine, but I ended up with a fatal error.
What really helped was to follow Felix' approach, who wrote in the
first mail of this thread:
Now, a second script, pasting first "slices" of 5*30° together,
and than merging all the slices...
This way, one can reduce the processing time to some 10-20% compared
to the "all in one run approach". Concerning the size of the
intermediate slices: the smaller they are, the faster it goes. The
final step of gdalbuilding a vrt from the imtermediate slices and
gdal_translating it to the final output.tif is also very fast.
In order to avoid NODATA gaps in the final output.tif: It seems to be
safer if the intermediate slices are slightly overlapping, rather
being than just adjacent.
Hermann
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
