Quoting Rahkonen Jukka <jukka.rahko...@mmmtike.fi>: > > > > Jukka, you can probably speed up considerably the gdalwarp > > process by adding "-co TILED=YES" to the gdalwarp > > commandline. For such a big output file, using the default > > stripped structure of a TIFF will be very inefficient because > > the way gdalwarp proceeds (by sort of tiles) causes a lot of > > cache thrashing and I/O. A little of tuning with the "-wm > > XXX", "-config GDAL_CACHEMAX YYY" and -multi options might also help. > > I can have a try. I want to change just on option at a time and now it > is running with -co TILED=YES option. I will see tomorrow how did it do. > Now I can say just that in the beginning things looked much better. > Memory consumption and processor load were much higher and disk unit > went more silently. But after some half an hour the conversion seems to > have almost stopped at 2,302,729,094 bytes output file size. Gdalwarp > reserves 390 megabytes of memory and utilises 5-15 percent of processor > time.
I have no explanation in mind why it seems to have stalled at 2.3 GB. I would let it go and see what happens a few hours after. > > Can you suggest any reasonable values for other parameters? This old > machine is running on Windows XP Pro and it has only 1 GB memory and one > 3 GHz processor. The -wm defaults to 64 MB and GDAL_CACHEMAX to 40 MB. So increasing both to 300 for example might work (it will let 1000-2*300=400 MB for the rest of the OS and other buffers of gdalwarp). If you see the computer starts swapping, those values should be reduced. See http://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp for a discussion on those parameters. > I do not have myself a real need for making large mosaics through .vrt > files at the moment, but as the set-up is ready and the computer > otherwise idle I can well let it run and gather some hopefully > comparable numbers. I have also a bit more powerful computer with 8 > cores that I can use for final run if there is a reason to believe that > gdalwarp can do the job with some settings. I guess -multi stands for > multithreading and it may suit well the 8-core system. -multi can also be helpfull with a single core system. It uses a thread for I/O and another one for computation. The I/O thread is mostly I/O bound and doesn't generally consumes much CPU, except if you use a compressed input or output file (which might be the case with your JP2K data). But having more than 2 cores won't help much. > > -Jukka- > _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
