(starting a new thread to avoid derailing the static-build one any further)
Totally agreed on the mismatch idea, but the code in question is all
self-contained down in *ogr/ogrsf_frmts/flatgeobuf* and the *flatbuffers*
sub-project (which is a snapshot of a Google OSS project) so I'm struggling
to see how there could be a mismatch.
Also, although we're building on CentOS 7, we're using relatively new
compilers (GCC 11.4 and Clang 14.0.6), and we bundle the matching newer
runtimes.
We don't have a full static build stack on our normal dev platform (Ubuntu
22.04) so I haven't been able to repro the problem there.
I should have mentioned the first time that we have tried using ASAN, and
it definitely catches something wrong, but the behavior is different, and
varies if you add more debug printfs. For example:
DEBUG: vector_downward::push() num = 16
DEBUG: about to reallocate, buf_ = 0, cur_ = 0, scratch = 0
DEBUG: reallocated, buf_ = 0x61900062d380, cur_ = 0x61900062cf80, scratch =
0
DEBUG: vector_downward::push() ptr = 0x61900062cf70, about to do memcpy
=================================================================
==25459==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x61900062cf70 at pc 0x7f8933eb87f6 bp 0x7fffa7aa0e70 sp 0x7fffa7aa0620
WRITE of size 16 at 0x61900062cf70 thread T0
...but it's still not obvious what exactly is going wrong. The code and
data flow makes perfect sense when you step through it in a dynamic build
that doesn't fail.
Like I said, the frustrating part is that a simple test program (attached)
compiled against the same set of static libs works fine.
S
On Tue, Feb 20, 2024 at 12:33 PM Robert Coup <robert.c...@koordinates.com>
wrote:
> Hi Simon,
>
> On Tue, 20 Feb 2024 at 18:58, Simon Eves via gdal-dev <
> gdal-dev@lists.osgeo.org> wrote:
>
>> We still have one VERY strange issue whereby FlatGeoBuf export fails in a
>> very consistent and reproducible form down in the flatbuffer code, but only
>> in the static build, and only in the full system. I have written a simple
>> test harness that links the very same static libgdal and does a simple GDAL
>> startup and FGB export of a single feature and that works fine. It's some
>> kind of data/stack corruption when it first tries to write to the
>> flatbuffer on the first feature, which results in a pointer member of the
>> buffer class becoming 0x100000000000 (always) instead of null, and then it
>> stops on an assert. There is also one private function in the
>> vector_downward class which the debugger won't even step into in that
>> build. I can even put printfs in that function and they don't come out.
>> I've tried it on CentOS and on Ubuntu, with GCC and Clang, and it's always
>> the same. Everything else in GDAL works just fine (we have LOTS of
>> import/export unit tests). This makes zero sense as all the FGB code is
>> internal to GDAL and compiled together. I've been poking at it for over a
>> week and it's doing my head in.
>>
>
> One cause of this sort of crash is a header/library mismatch somewhere
> where a function is expecting different parameters/types than the caller is
> actually providing. Otherwise, maybe a bug in glibc/libstdc++/gcc/something
> that's been fixed in the intervening ten years since CentOS 7 was released?
>
>
> If you run your *build* on a modern distro/libc/gcc/etc does it change
> things? If it's the same, maybe hints more towards the former.
>
> ASAN (https://github.com/google/sanitizers/wiki/AddressSanitizer) might
> help track down stack/heap corruption.
>
> Rob :)
>
>
#include <gdal.h>
#include <gdal_priv.h>
#include <ogrsf_frmts.h>
#include <iostream>
#include <array>
#include <string>
#include <assert.h>
#define GDAL_DATA_PATH DEPS "/share/gdal"
#define PROJ_DATA_PATH DEPS "/share/proj"
void gdal_error_handler(CPLErr err_class, int err_no, const char* err_msg) {
static constexpr std::array<const char*, 5> err_class_strings{
"Info",
"Debug",
"Warning",
"Failure",
"Fatal",
};
std::string log_msg = std::string("GDAL ") + err_class_strings[err_class] + ": " +
err_msg + " (" + std::to_string(err_no) + ")";
std::cout << log_msg << std::endl;
}
int main(int argc, char** argv)
{
// parameters
if (argc != 3) {
std::cout << "Usage: gdal_export_test <output_file> <format>" << std::endl;
std::cout << "<format> must be one of the names returned by ogrinfo --formats, e.g. GeoJSON, FlatGeobuf, \"ESRI Shapefile\"" << std::endl;
exit(1);
}
const char* output_file = argv[1];
const char* driver_name = argv[2];
// init GDAL
std::cout << "Initializing GDAL" << std::endl;
std::cout << "GDAL_DATA_PATH = " << GDAL_DATA_PATH << std::endl;
std::cout << "PROJ_DATA_PATH = " << PROJ_DATA_PATH << std::endl;
setenv("GDAL_DATA", GDAL_DATA_PATH, true);
setenv("PROJ_DATA", PROJ_DATA_PATH, true);
GDALAllRegister();
OGRRegisterAll();
CPLSetErrorHandler(*gdal_error_handler);
// get driver
std::cout << "Getting driver" << std::endl;
auto* gdal_driver = GetGDALDriverManager()->GetDriverByName(driver_name);
assert(gdal_driver);
// open dataset
std::cout << "Opening dataset" << std::endl;
auto* gdal_dataset = gdal_driver->Create(output_file, 0, 0, 0, GDT_Unknown, nullptr);
assert(gdal_dataset);
// create spatial reference
std::cout << "Creating spatial reference" << std::endl;
OGRSpatialReference spatial_reference;
spatial_reference.importFromEPSG(4326);
spatial_reference.SetAxisMappingStrategy(OAMS_TRADITIONAL_GIS_ORDER);
// create layer
std::cout << "Creating layer" << std::endl;
auto const geometry_type = wkbPoint;
auto* ogr_layer = gdal_dataset->CreateLayer("mylayer", &spatial_reference, geometry_type, nullptr);
assert(ogr_layer);
// create a field
OGRFieldDefn field_defn("myint", OFTInteger);
ogr_layer->CreateField(&field_defn);
// make a point feature
std::cout << "Creating point feature" << std::endl;
auto* ogr_feature = OGRFeature::CreateFeature(ogr_layer->GetLayerDefn());
assert(ogr_feature);
auto* ogr_geometry = OGRGeometryFactory::createGeometry(OGRwkbGeometryType::wkbPoint);
assert(ogr_geometry);
auto* ogr_point = dynamic_cast<OGRPoint*>(ogr_geometry);
assert(ogr_point);
ogr_point->setX(3.1415926);
ogr_point->setY(1.2345678);
ogr_feature->SetGeometry(ogr_geometry);
// add field value
const int field_index = 0;
auto const field_type = ogr_feature->GetFieldDefnRef(field_index)->GetType();
ogr_feature->SetField(field_index, 42);
// add feature to layer
std::cout << "Adding feature to layer" << std::endl;
assert(ogr_layer->CreateFeature(ogr_feature) == OGRERR_NONE);
// destroy feature
std::cout << "Destroying feature" << std::endl;
OGRFeature::DestroyFeature(ogr_feature);
// close dataset
std::cout << "Closing dataset" << std::endl;
GDALClose(gdal_dataset);
// done
std::cout << "Done" << std::endl;
exit(0);
}_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
