Re: [gdal-dev] gdal_merge is very slow

2010-07-18 Thread Greg Coats
On Jul 18, 2010, at 6:09 PM, Leith Bade wrote: > they only have a 32-bit build Not true. The complaint that Kakadu has never offered 64 bit versions of their command line executables for free is false. The Kakadu web site now offers, for free, version 6.4 for Mac OS X compiled for 32 bit and 64

Re: [gdal-dev] gdal_merge is very slow

2010-07-18 Thread Leith Bade
Unfortunately I can't afford a Kakadu license for personal use so I am using the precompiled demo binaries. ( http://www.kakadusoftware.com/index.php?option=com_content&task=view&id=26&Itemid=22 ) As you can see they only have a 32-bit build (and it is not even compiled as Large Address Aware http

Re: [gdal-dev] gdal_merge is very slow

2010-07-18 Thread Greg Coats
On Jul 18, 2010, at 12:44 AM, Leith Bade wrote: > kdu_compress also has a memory limit at 2 GB Not true. Based on your comment, I suspect that you are using a version of Kakadu that is more than 3 years old, or that was deliberately compiled for a 32 bit environment. Since version 6.0, releas

Re: [gdal-dev] gdal_merge is very slow

2010-07-18 Thread Even Rouault
Leith, For the record, I've just made changes in the development version that allow specifying more than 2 GB (the effective value might be 2GB or 3GB on 32 bit operating systems). See http://trac.osgeo.org/gdal/ticket/3689 And the current limit is 2047, not 2147. (2147 * 1024 * 1024 is a negat

Re: [gdal-dev] gdal_merge is very slow

2010-07-17 Thread Leith Bade
That seemed to do the trick. It finished overnight, a lot faster than before. Final file was 83GB. (Only 1/2 the dataset was used) Is there a reason GDAL_CACHEMAX 2147 is the maximum? Under 64-bit a 32-bit process has 4GB of virtual memory rather than 2GB under 32-bit (or 3GB with the /3GB Window

Re: [gdal-dev] gdal_merge is very slow

2010-07-16 Thread Greg Coats
Since you have 4 GB RAM, I suggest overriding the 40 MB GDAL_CACHEMAX default, and increasing it gdal_merge.py --config GDAL_CACHEMAX 1000 The maximum possible GDAL_CACHEMAX is gdal_merge.py --config GDAL_CACHEMAX 2147 Greg On Jul 16, 2010, at 9:19 PM, Leith Bade wrote: > Hi, > > I am trying t

Re: [gdal-dev] gdal_merge is very slow

2010-07-16 Thread Leith Bade
OK I will try the tiling. The aim of this merge is to generate an input for creating a compressed JPEG 2000 image (either using GDAL, Kakadu tools or the ER Image Compressor). In the meantime I have installed the 30-day demo of ER Mapper. This program was able to generate a mosaic in only a few s

Re: [gdal-dev] gdal_merge is very slow

2010-07-16 Thread Greg Coats
I do not see that you specified that the output TIFF image be tiled -co TILED=YES -co BLOCKXSIZE=512 -co BLOCKYSIZE=512 gdal_merge.py supports the -v input option that reports progress as a % for every source image merged into the destination image. It is better to do the initial gdal_merge.py un

[gdal-dev] gdal_merge is very slow

2010-07-16 Thread Leith Bade
Hi, I am trying to use gdal_merge to mosaic a very large topo GeoTIFF set. Uncompressed the data set is 60GB, but I keep it stored with DEFLATE compression which results in a dataset under 10GB. Mosaicked the uncompressed file will be 125GB because of the large regions of nodata generated. Unfor