Hello all,
I'm trying to build GDAL python wheels and am having some autotest fails that
is preventing the build. I think most if not all of these fails are due to
rtree not being available in sqlite. AFAIK I've done everything needed but the
build recipe must be missing something.
The dockerfile contents are below and I run it with the command
`DOCKER_BUILDKIT=0 docker build -f Dockerfile -t gdal-wheelbuilder`. It gets as
far as `ctest -V` then exits with the following test fails:
46 - autotest_gdrivers (Failed)
48 - autotest_ogr (Failed)
50 - autotest_pyscripts (Failed)
Thanks,
Jon
FROM quay.io/pypa/manylinux_2_28_x86_64
# Arrow install instructions from https://arrow.apache.org/install/
RUN dnf update -y \
&& dnf groupinstall -y "Development Tools" \
&& dnf install -y epel-release \
&& dnf install -y
https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(cut -d: -f5
/etc/system-release-cpe | cut -d. -f1).noarch.rpm \
&& dnf install -y
https://packages.apache.org/artifactory/arrow/almalinux/$(cut -d: -f5
/etc/system-release-cpe | cut -d. -f1)/apache-arrow-release-latest.rpm \
&& dnf config-manager --set-enabled epel \
&& dnf config-manager --set-enabled powertools \
&& dnf install -y arrow-devel \
&& dnf install -y arrow-dataset-devel \
&& dnf install -y parquet-devel
RUN dnf install -y \
ant \
autoconf \
automake \
bash-completion \
cmake \
ca-certificates \
ccache \
clang \
cmake \
curl-devel \
doxygen \
git \
gnupg \
gpsbabel \
lcov \
hdf5-devel \
json-devel \
json-c-devel \
libaec-devel \
libdeflate-devel \
libjpeg-devel \
libkml-devel \
libpng-devel \
libpq-devel \
libtiff-devel \
libtool \
libwebp-devel \
libxml2-devel \
libzstd-devel \
make \
minizip-devel \
numactl \
openssl-devel \
openjpeg2-devel \
pkg-config \
poppler-devel \
postgresql \
postgresql-contrib \
python3-devel \
python38-devel \
python3-lxml \
python3-numpy \
python3-pip \
python3-pytest \
python3-setuptools \
python3-virtualenv \
swig \
unzip \
wget \
zip \
zlib-devel
# Install geos
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://download.osgeo.org/geos/geos-3.13.0.tar.bz2 \
&& tar jxf geos-3.13.0.tar.bz2 \
&& cd /src/geos-3.13.0 \
&& mkdir build \
&& cd build \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_TESTING=OFF
-DCMAKE_BUILD_TYPE=Release .. \
&& make -j$(nproc) \
&& make install \
&& cd / \
&& rm -rf /src
# Build a SQLite version with -DSQLITE_TRUSTED_SCHEMA=0
RUN mkdir sqlite \
&& wget -q https://sqlite.org/2023/sqlite-autoconf-3430100.tar.gz -O - \
| tar xz -C sqlite --strip-components=1 \
&& cd sqlite \
&& CFLAGS="-O2 -DSQLITE_TRUSTED_SCHEMA=0 -DSQLITE_ENABLE_COLUMN_METADATA
-DSQLITE_ENABLE_RTREE" ./configure
--prefix=/usr/local/install-sqlite-trusted-schema-off \
&& make -j$(nproc) \
&& make -j$(nproc) install \
&& cd .. \
&& rm -rf sqlite
# Fix proj version
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://download.osgeo.org/proj/proj-9.1.1.tar.gz \
&& tar xzf proj-9.1.1.tar.gz \
&& cd /src/proj-9.1.1 \
&& mkdir build \
&& cd build \
&& cmake
-DEXE_SQLITE3=/usr/local/install-sqlite-trusted-schema-off/bin/sqlite3 .. \
&& cmake --build . \
&& cmake --build . --target install \
&& rm -rf /src
# Build libgeotiff (system libgeotiff pulls in system libproj as a dependency)
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O
https://github.com/OSGeo/libgeotiff/releases/download/1.7.3/libgeotiff-1.7.3.tar.gz
\
&& tar xzf libgeotiff-1.7.3.tar.gz \
&& cd libgeotiff-1.7.3 \
&& mkdir build_autoconf \
&& cd build_autoconf \
&& export LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/usr/lib:/lib \
&& CFLAGS="-Wall -Wextra -Werror" ../configure \
&& make -j$(nproc) \
&& make check \
&& cd .. \
&& mkdir build_cmake \
&& cd build_cmake \
&& cmake -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_C_FLAGS="-Wall -Wextra
-Werror" .. \
&& make -j$(nproc) \
&& rm -rf /src
# Build freexl (spatialite dependency)
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://www.gaia-gis.it/gaia-sins/freexl-2.0.0.tar.gz \
&& tar xzf freexl-2.0.0.tar.gz \
&& cd /src/freexl-2.0.0 \
&& ./configure \
&& make -j$(nproc) \
&& make install \
&& rm -rf /src
# Build spatialite so it uses correct PROJ and sqlite3
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O
https://www.gaia-gis.it/gaia-sins/libspatialite-sources/libspatialite-5.1.0.tar.gz
\
&& tar xzf libspatialite-5.1.0.tar.gz \
&& cd /src/libspatialite-5.1.0 \
&& SQLITE3_CFLAGS="-I/usr/local/install-sqlite-trusted-schema-off/include/"
SQLITE3_LIBS="-Wl,--enable-new-dtags,-rpath=/usr/local/install-sqlite-trusted-schema-off/lib
-L/usr/local/install-sqlite-trusted-schema-off/lib -lsqlite3" ./configure
--disable-rttopo \
&& make -j$(nproc) \
&& make install \
&& rm -rf /src
# configure shared libraries
RUN ldconfig -v
# Add GDAL source to container
# and checkout current branch and commit (see README.md).
# Also patch in commits
https://github.com/OSGeo/gdal/commit/b36072fa1539b8ac97ea7864071d3468ac5b49ab
# and
https://github.com/OSGeo/gdal/commit/222e1c111437f9fd6e43eb384f0903684735750b
to get rid of
# arrow/parquet deprecation warnings
ARG GITHUB_WORKSPACE=/src
RUN mkdir -p $GITHUB_WORKSPACE \
&& cd $GITHUB_WORKSPACE \
&& git config --global user.email y...@example.com<mailto:y...@example.com>
\
&& git config --global user.name "Your Name" \
&& git clone -b release/3.10 https://github.com/OSGeo/gdal.git superbuild \
&& cd superbuild \
&& git checkout 9b7a7c8 \
&& git cherry-pick b36072fa \
&& git cherry-pick 222e1c11
# Install test requirements
RUN python3.8 -m pip install -U pip setuptools numpy importlib_metadata \
&& python3.8 -m pip install -r
$GITHUB_WORKSPACE/superbuild/autotest/requirements.txt
## This section based on build-linux-ubuntu-focal job in
.github/workflows/cmake_builds.yml ##
ARG CMAKE_UNITY_BUILD=OFF
# - name: Configure
RUN set -eu \
&& cd $GITHUB_WORKSPACE/superbuild \
&& export CXXFLAGS="-march=native -O2 -Wodr -flto-odr-type-merging -Werror"
\
&& export CFLAGS="-O2 -march=native -Werror" \
&& mkdir build \
&& cd build \
&& cmake .. \
-Werror=dev \
-DCMAKE_INSTALL_PREFIX=$GITHUB_WORKSPACE/install-gdal \
-DCMAKE_UNITY_BUILD=${CMAKE_UNITY_BUILD} \
-DGDAL_USE_PUBLICDECOMPWT:BOOL=ON \
-DPUBLICDECOMPWT_URL=https://github.com/rouault/PublicDecompWT \
-DWERROR_DEV_FLAG="-Werror=dev" \
-DPython_LOOKUP_VERSION=3.8 \
-DUSE_CCACHE=OFF \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON \
-DGDAL_USE_TIFF_INTERNAL=ON \
-DGDAL_USE_GEOTIFF_INTERNAL=ON \
-DSQLite3_INCLUDE_DIR=/usr/local/install-sqlite-trusted-schema-off/include \
-DSQLite3_LIBRARY=/usr/local/install-sqlite-trusted-schema-off/lib/libsqlite3.so
\
-DBUILD_CSHARP_BINDINGS:BOOL=OFF \
&& unset CXXFLAGS \
&& unset CFLAGS
# - name: Build
RUN GIT_LFS_SKIP_SMUDGE=1 cmake --build $GITHUB_WORKSPACE/superbuild/build --
-j$(nproc)
# - name: test (with command targets)
RUN cmake --build $GITHUB_WORKSPACE/superbuild/build --target quicktest --
-j$(nproc)
# - name: test (with ctest)
RUN cd $GITHUB_WORKSPACE/superbuild/build \
&& ctest -V
# - name: install
RUN cmake --build $GITHUB_WORKSPACE/superbuild/build --target install --
-j$(nproc)
## END build-linux-ubuntu-focal job in .github/workflows/cmake_builds.yml ##
Jon Morris
Software Developer
--
e: jon.mor...@jbarisk.com
d:+44 (0)1756 587229
t: +44 (0)1756 799919
www.jbarisk.com
All JBA Risk Management's email messages contain confidential information and
are intended only for the individual(s) named. If you are not the named
addressee you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately by email if you have received this email
by mistake and delete this email from your system.
JBA Risk Management Limited is registered in England, company number 07732946,
1 Broughton Park, Old Lane North, Broughton, Skipton, North Yorkshire, BD23
3FD, England.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
