Adam Nowacki wrote:
You can test with http://pastebin.com/m45e46f53
Also remember to use -wo SKIP_NOSOURCE=YES
Now the input images are read in many chunks, rather than 1 (or max 2)
before. Nevertheless, the performance is really great. Now, 100 tiles
are warped in 5 minutes! That's 4-5 times faster than anything else I
tried before.
Hermann
Before applying the patch, with intentionally small memory
(-wm 40, which was fastest, as far as I could see)
GDAL: GDALOpen(unzipped/ASTGTM_N30E039_dem.tif, this=0x9711470) succeeds
as GTiff.
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=0,0,3601x2185 Dst=42437,38906,3031x2594
GDAL: Potential thrashing on band 1 of unzipped/ASTGTM_N30E039_dem.tif.
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=636,0,2965x3601 Dst=45468,38906,3032x2594
GDAL: GDALClose(unzipped/ASTGTM_N30E039_dem.tif, this=0x9711470)
After applying the patch, with -wm 300 --config GDAL_CACHEMAX 300
(and -wo SKIP_NOSOURCE=YES, which I always used)
GDAL: GDALOpen(unzipped/ASTGTM_N30E039_dem.tif, this=0x86f5dd0) succeeds
as GTiff.
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=704,0,435x91 Dst=45278,40770,95x81
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=1042,0,435x207 Dst=45373,40770,95x81
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=0,0,1383x2185 Dst=45089,40851,379x649
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=1380,0,863x439 Dst=45468,40689,189x162
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=2051,0,1058x673 Dst=45657,40527,190x324
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=2725,0,876x908 Dst=45847,40527,189x324
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=3585,167,16x368 Dst=46036,40608,95x81
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=3490,413,111x367 Dst=46036,40689,95x81
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=3395,659,206x367 Dst=46036,40770,95x81
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=636,204,2965x2911 Dst=45468,40851,758x649
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=3592,2125,9x366 Dst=46226,41175,94x81
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=3498,2371,103x366 Dst=46226,41256,94x81
GDAL: GDALWarpKernel()::GWKGeneralCase()
Src=3309,2617,292x615 Dst=46226,41337,94x163
GDAL: GDALClose(unzipped/ASTGTM_N30E039_dem.tif, this=0x86f5dd0)
Hermann Peifer wrote:
Adam Nowacki wrote:
Some rather counterintuitive gdalwarp behavior: the bigger
dfWarpMemoryLimit (-wm setting) the more cpu time will be wasted on
warping not existing pixels. Why? Warping begins with the destination
window size of entire output image size. If this size is larger than
dfWarpMemoryLimit it is split in half along the longest axis and any
half that doesn't contain the currently processed source file is
discarded. With large dfWarpMemoryLimit this subdivision process will
stop early with still large portions of out of source image pixels.
Given Adam's explanations above, could someone tell me if my below
assumptions are correct? Thanks.
Just to repeat, the input files are ASTER GTiffs in WGS84, 3601x3601
pixels each. The output is a single mosaic of all tiles, a 100m GTiff,
in LAEA projection
gdalwarp -wm 300 --config GDAL_CACHEMAX 300 --debug on ... reports:
Src=0,0,3601x3601 Dst=36375,31125,12125x10375
I understand this as follows: the source image of 3601x3601 pixels is
read as one chunk (which after reprojection and resampling should be
around 800x1100 pixels). However, 12125x10375 pixels are actually
written to the output file. So 99.5% of the destination window must be
completely unrelated to my input image.
If I counter-intuitively reduce the memory limit to, say: 40 MB and
leave anything else unchanged, then gdalwarp behaviour changes to
Src=0,0,3601x3601 Dst=45468,38906,1516x2594
This tells me that the destination window is much smaller and gdalwarp
will not waste time to write out so many irrelevant pixels, correct?
Perhaps gdalwarp should not only test if the destination window fits
into memory, but also check what would be the minimum destination
window for warping the input image. This could speed up the mosaicing
of small tiles into a bigger output file.
Hermann
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
