Thanks Even, that’s what I suspected.
Best,
Jesse
Lead Computer Scientist
Science Systems and Applications, Inc.
Dr Compton Tucker Team
NASA Goddard Space Flight Center
From: Even Rouault <even.roua...@spatialys.com>
Date: Thursday, July 3, 2025 at 4:23 PM
To: Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC]
<jesse.r.me...@nasa.gov>, gdal dev <gdal-dev@lists.osgeo.org>
Subject: [EXTERNAL] Re: [gdal-dev] Possible to issue sqlite3_limit with
"memory" OGR driver?
CAUTION: This email originated from outside of NASA. Please take care when
clicking links or opening attachments. Use the "Report Message" button to
report suspicious messages to the NASA SOC.
Jesse,
Maybe you use SQL with the SQLite dialect? Otherwise we lack context to tell
why you get SQLite kicking it.
I don't think sqlite3_limit() would really help you here, givenit can be used
to grow settings beyond the compile time default which is 1 GiB I believe
("Attempts to increase a limit above its hard upper bound are silently
truncated to the hard upper bound."). You would need to recompile SQLite itself.
Even
Le 03/07/2025 à 22:14, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND
APPLICATIONS INC] via gdal-dev a écrit :
We’re trying to gdal.Polygonize a feature that exceeds SQLite’s default blob
size of 1,000,000,000 bytes. Curiously, we’re using the memory dataset, but
this appears to leverage SQLite under the hood (evidenced by the error message:
ERROR 1: In GetNextRawFeature(): sqlite3_step() : string or blob too big).
SQLite docs state the that limit can be changed at runtime using the
sqlite3_limit<https://www.sqlite.org/c3ref/limit.html>(db,SQLITE_LIMIT_LENGTH<https://www.sqlite.org/c3ref/c_limit_attached.html#sqlitelimitlength>,...)
interface. Maybe there could be some way to leverage PRELUDE_STATEMENTS to
achieve this but we are not “opening” a dataset - we’re creating one directly
from the memory driver.
What is the best way to work around this?
Best,
Jesse
Lead Computer Scientist
Science Systems and Applications, Inc.
Dr Compton Tucker Team
NASA Goddard Space Flight Center
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com<http://www.spatialys.com/>
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
