Hi Even,
I am using GDAL 2.4.4. I may try the latest version as you suggested.
The NITF file size is 4,294,967,295 bytes
The vendor used a non-GDAL implementation, and was able to display all
image.
Metadata:
NITF_ABPP=16
NITF_BLOCKA_BLOCK_COUNT=01
NITF_BLOCKA_BLOCK_INSTANCE_01=01
NITF_BLOCKA_FRFC_LOC_01=
NITF_BLOCKA_FRLC_LOC_01=
NITF_BLOCKA_LAYOVER_ANGLE_01=000
NITF_BLOCKA_LRFC_LOC_01=
NITF_BLOCKA_LRLC_LOC_01=
NITF_BLOCKA_L_LINES_01=19694
NITF_BLOCKA_N_GRAY_01=00000
NITF_BLOCKA_SHADOW_ANGLE_01=180
NITF_CCS_COLUMN=0
NITF_CCS_ROW=0
NITF_CLEVEL=07
NITF_ENCRYP=0
NITF_FBKGC= 0, 0, 0
NITF_FDT=
NITF_FHDR=NITF02.10
NITF_FSCATP=
NITF_FSCAUT=
NITF_FSCLAS=U
NITF_FSCLSY=CA
NITF_FSCLTX=
NITF_FSCODE=
NITF_FSCOP=00000
NITF_FSCPYS=00000
NITF_FSCRSN=
NITF_FSCTLH=
NITF_FSCTLN=
NITF_FSDCDT=
NITF_FSDCTP=
NITF_FSDCXM=
NITF_FSDG=
NITF_FSDGDT=
NITF_FSREL=
NITF_FSSRDT=
NITF_FTITLE=
NITF_IALVL=0
NITF_IC=NC
NITF_ICAT=SAR
NITF_ICORDS=G
NITF_IDATIM=
NITF_IDLVL=
NITF_IGEOLO=
NITF_IID1=P1
NITF_IID2=
NITF_ILOC_COLUMN=0
NITF_ILOC_ROW=0
NITF_IMAG=1.0
NITF_IMODE=P
NITF_IREP=MULTI
NITF_ISCATP=
NITF_ISCAUT=
NITF_ISCLAS=U
NITF_ISCLSY=CA
NITF_ISCLTX=
NITF_ISCODE=
NITF_ISCRSN=
NITF_ISCTLH=
NITF_ISCTLN=
NITF_ISDCDT=
NITF_ISDCTP=
NITF_ISDCXM=
NITF_ISDG=
NITF_ISDGDT=
NITF_ISORCE=
NITF_ISREL=
NITF_ISSRDT=
NITF_ONAME=
NITF_OPHONE=
NITF_OSTAID=CSA
NITF_PJUST=R
NITF_PVTYPE=INT
NITF_STYPE=BF01
NITF_TGTID=
------------------------------------------------------------------------
*From:* Even Rouault <even.roua...@spatialys.com>
*Sent:* April 7, 2021 5:23 AM
*To:* ni hao <ni_ha...@hotmail.com>; gdal-dev@lists.osgeo.org
<gdal-dev@lists.osgeo.org>
*Subject:* Re: [gdal-dev] NITF Int16 image size limit
Hi,
Actually the error message doesn't look like there's a uint32 overflow
as it displays a file offset that is beyond the 4 GB limit.
Did you check that your file size is at least 4324329780 + 4194302 = 4
328 524 082 bytes ? I guess it is not, or then there's some mysterious
error in the low level GDAL I/O layer.
So if your file is smaller than that,
- either it is corrupted (would be good if you could check that with a
non-GDAL implementation)
- or there's a bug in the NITF driver in how it computes offsets.
Could you report the values of the NITF_ABPP, NITF_IC and NITF_IMODE
metadata reported by gdalinfo ?
Even
Le 07/04/2021 à 02:09, ni hao a écrit :
gdalinfo output i:
Driver: NITF/National Imagery Transmission Format
Files: ##CH_CV_GRD\imagery\12499.ntf
Size is 61180, 19694
Coordinate System is `'
GCP Projection =
GEOGCS["WGS 84", ... ]
Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
Band 2 Block=1024x1024 Type=UInt16, ColorInterp=Gray
Also I am sure it is the UInt16 overflow, as the message is:
ERROR 3: Unable to read 4194302 byte block from 4324329780
and 2**32 = 4294967296
------------------------------------------------------------------------
*From:* br...@frogmouth.net <mailto:br...@frogmouth.net>
<br...@frogmouth.net> <mailto:br...@frogmouth.net>
*Sent:* April 6, 2021 8:17 PM
*To:* Even Rouault <even.roua...@spatialys.com>
<mailto:even.roua...@spatialys.com>; ni hao <ni_ha...@hotmail.com>
<mailto:ni_ha...@hotmail.com>
*Cc:* gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>
<gdal-dev@lists.osgeo.org> <mailto:gdal-dev@lists.osgeo.org>
*Subject:* Re: [gdal-dev] NITF Int16 image size limit
Also if you can show the headers (e.g. from gdalinfo), that might
provide indication of exactly which value is the issue.
Brad
On 7 Apr. 2021 04:00, Even Rouault <even.roua...@spatialys.com>
<mailto:even.roua...@spatialys.com> wrote:
Please keep the mailing list copied (I re-added it)
This is not the most recent one (3.2.2 is), but skimming quickly
through release notes, I can't see anything directly related, so
it might still be current with the recent versions. Altough I see
that a bunch of "avoid unsigned integer overflow" type of fixes
have been done since 2.4, so some of them might have accidentally
fixed the issue.
If you can try with the latest version ( Docker images mentioned
at https://github.com/OSGeo/gdal/tree/master/gdal/docker
<https://github.com/OSGeo/gdal/tree/master/gdal/docker>, Conda,
etc .), that could be interesting.
Le 06/04/2021 à 19:51, ni hao a écrit :
Hi Even,
The version is 2.4.4.
------------------------------------------------------------------------
*From:* Even Rouault <even.roua...@spatialys.com>
<mailto:even.roua...@spatialys.com>
*Sent:* April 6, 2021 2:46 PM
*To:* ni hao <ni_ha...@hotmail.com>
<mailto:ni_ha...@hotmail.com>; gdal-dev@lists.osgeo.org
<mailto:gdal-dev@lists.osgeo.org> <gdal-dev@lists.osgeo.org>
<mailto:gdal-dev@lists.osgeo.org>
*Subject:* Re: [gdal-dev] NITF Int16 image size limit
Shawn,
This sounds more like a unwanted integer overflow somewhere
in the NITF driver. Is this with a recent GDAL ? If so,
please file a bug at https://github.com/OSGeo/gdal/issues/new
<https://github.com/OSGeo/gdal/issues/new> with all the
details needed. A link to the dataset would be ideal, but
otherwise please provide in the ticket description the output
of "gdalinfo your.nitf"
Even
Le 06/04/2021 à 19:35, ni hao a écrit :
Hi list,
I encountered problem ingesting a large NITF Int16 image
with GDAL:
NITF 2-band image in Int16 format, 4.3 GB. It has 19690
lines x 61180 pixels. The upper 90% of the image looks
fine. But the last 2000 lines fail to load by GDAL.
That coincides with the 32-bit boundary.
Note that another larger NITF in 32-bit float, 10.1 GB
file loads fine.
Is there a hard limit on NITF Int16 file size?
Thanks,
Shawn
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
<https://lists.osgeo.org/mailman/listinfo/gdal-dev>
--
http://www.spatialys.com <http://www.spatialys.com>
My software is free, but my time generally not.
--
http://www.spatialys.com <http://www.spatialys.com>
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
<https://lists.osgeo.org/mailman/listinfo/gdal-dev>
--
http://www.spatialys.com <http://www.spatialys.com>
My software is free, but my time generally not.
