I am trying to use Conda as a package manager for isolated C++ development environments. But unfortunately, when using CMake with the Anaconda-provided compilers [1] (which are used to compile the binary packages in the Anaconda repositories), things do not work as expected.
I have a small test case available here [2], with an executable calling a shared library and a third-party dependency installed with Conda. [1]: https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html [2]: https://gist.github.com/smancill/b28ca07ac11fdf285b4d559545a1630b -------------------------------------------------- First, when using the system compiler, all works fine (but I am not sure of the binary compatibility with the Conda packages, that's why I want to use the Anaconda compilers): # create the environment and install XercesC $ conda create -n test-system xerces-c ... environment location: /Users/smancill/.local/share/miniconda3/envs/test-system ... The following NEW packages will be INSTALLED: icu: 58.2-h4b95b61_1 libcxx: 4.0.1-h579ed51_0 libcxxabi: 4.0.1-hebd6815_0 xerces-c: 3.2.1-h44e365a_0 ... # activate the environment $ conda activate test-system $ mkdir build-osx-system $ cd build-osx-system $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DCMAKE_PREFIX_PATH=$CONDA_PREFIX .. -- The CXX compiler identification is AppleClang 9.0.0.9000039 -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works ... -- Found XercesC: /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib (found version "3.2.1") -- Configuring done -- Generating done -- Build files have been written to: /Users/smancill/src/conda-test/build-osx-system $ make -j1 VERBOSE=1 ... [100%] Linking CXX executable bar /usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt --verbose=1 /usr/bin/c++ -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk -mmacosx-version-min=10.12 -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/bar.dir/bar.cpp.o -o bar -Wl,-rpath,/Users/smancill/src/conda-test/build-osx-system -Wl,-rpath,/Users/smancill/.local/share/miniconda3/envs/test-system/lib libfoo.dylib /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib ... The build directory (~/src/conda-test/build-osx-system) and the conda environment lib directory (~/.local/share/miniconda3/envs/test-system/lib) are correctly added to the RPATH in the build tree by the link command: $ ./bar Hello, world! $ otool -L ./bar ./bar: @rpath/libfoo.dylib (compatibility version 1.0.0, current version 0.0.0) @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0) $ otool -l ./bar | grep -A2 LC_RPATH cmd LC_RPATH cmdsize 56 path /Users/smancill/src/conda-test/build-osx-system (offset 12) -- cmd LC_RPATH cmdsize 80 path /Users/smancill/.local/share/miniconda3/envs/test-system/lib (offset 12) If I install the binary, it fails because I haven't configured CMake to set the install RPATH: $ make install ... Install the project... -- Install configuration: "" -- Installing: /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libfoo.dylib -- Installing: /Users/smancill/.local/share/miniconda3/envs/test-system/include/foo.hpp -- Installing: /Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar $ bar dyld: Library not loaded: @rpath/libfoo.dylib Referenced from: /Users/smancill/.local/share/miniconda4/envs/test-system/bin/bar Reason: image not found [1] 84611 abort bar $ otool -L $CONDA_PREFIX/bin/bar /Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar: @rpath/libfoo.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0) $ otool -l $CONDA_PREFIX/bin/bar | grep -A2 LC_RPATH # empty # deactivate the environment to start again $ conda deactivate The same can be observed on Linux. Everything works as it should. -------------------------------------------------- If I try to use Anaconda compilers on macOS, the build RPATH is not set properly anymore: # create the environment and install the Anaconda compiler for macOS, and XercesC $ conda create -n test-conda clangxx_osx-64 xerces-c ... environment location: /Users/smancill/.local/share/miniconda3/envs/test-conda ... The following NEW packages will be INSTALLED: cctools: 895-h7512d6f_0 clang: 4.0.1-h662ec87_0 clang_osx-64: 4.0.1-h1ce6c1d_11 clangxx: 4.0.1-hc9b4283_0 clangxx_osx-64: 4.0.1-h22b1bf0_11 compiler-rt: 4.0.1-h5487866_0 icu: 58.2-h4b95b61_1 ld64: 274.2-h7c2db76_0 libcxx: 4.0.1-h579ed51_0 libcxxabi: 4.0.1-hebd6815_0 llvm: 4.0.1-hc748206_0 llvm-lto-tapi: 4.0.1-h6701bc3_0 xerces-c: 3.2.1-h44e365a_0 ... # activate the environment (which sets the variables to use the Anaconda compiler) $ conda activate test-conda $ mkdir build-osx-conda $ cd build-osx-conda $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DCMAKE_PREFIX_PATH=$CONDA_PREFIX .. -- The CXX compiler identification is Clang 4.0.1 -- Check for working CXX compiler: /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++ -- Check for working CXX compiler: /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++ -- works ... -- Found XercesC: /Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib (found version "3.2.1") -- Configuring done -- Generating done -- Build files have been written to: /Users/smancill/src/conda-test/build-osx-conda $ make -j1 VERBOSE=1 ... [100%] Linking CXX executable bar /usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt --verbose=1 /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++ -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk -mmacosx-version-min=10.12 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs CMakeFiles/bar.dir/bar.cpp.o -o bar -Wl,-rpath,/Users/smancill/src/conda-test/build-osx-conda libfoo.dylib /Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib ... You can see that the environment lib path is not added to the RPATH by the link command, which in turns result in the executable not running from the build tree anymore: $ ./bar dyld: Library not loaded: @rpath/libxerces-c-3.2.dylib Referenced from: /Users/smancill/src/conda-test/build-osx-conda/./bar Reason: image not found [1] 89350 abort ./bar $ otool -L ./bar ./bar: @rpath/libfoo.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0) $ otool -l ./bar | grep -A2 LC_RPATH cmd LC_RPATH cmdsize 56 path /Users/smancill/src/conda-test/build-osx-conda (offset 12) # deactivate the environment $ conda deactivate -------------------------------------------------- If I try the Anaconda compilers on Linux, there are strange results too: # create the environment and install the Anaconda compiler for Linux, and XercesC $ conda create -n test-conda gxx_linux-64 xerces-c ... environment location: /home/vagrant/miniconda3/envs/test-conda ... The following NEW packages will be INSTALLED: binutils_impl_linux-64: 2.28.1-had2808c_3 binutils_linux-64: 7.2.0-had2808c_27 gcc_impl_linux-64: 7.2.0-habb00fd_3 gcc_linux-64: 7.2.0-h550dcbe_27 gxx_impl_linux-64: 7.2.0-hdf63c60_3 gxx_linux-64: 7.2.0-h550dcbe_27 icu: 58.2-h9c2bf20_1 libgcc-ng: 7.2.0-hdf63c60_3 libstdcxx-ng: 7.2.0-hdf63c60_3 xerces-c: 3.2.1-hac72e42_0 # activate the environment (which sets the variables to use the Anaconda compiler) $ conda activate test-conda $ mkdir build-linux-conda $ cd build-linux-conda $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DCMAKE_PREFIX_PATH=$CONDA_PREFIX .. -- The CXX compiler identification is GNU 7.2.0 -- Check for working CXX compiler: /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++ -- Check for working CXX compiler: /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++ -- works ... -- Found XercesC: /home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so (found version "3.2.1") -- Configuring done -- Generating done -- Build files have been written to: /vagrant/conda-test/build-linux-conda $ make -j1 VERBOSE=1 ... [100%] Linking CXX executable bar /usr/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt --verbose=1 /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++ -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now CMakeFiles/bar.dir/bar.cpp.o -o bar libfoo.so /home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so -Wl,-rpath,/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib: ... You can see that the environment lib path is added to the RPATH by the link command (unlike macOS): $ ./bar Hello World! But when inspecting the dependencies, the environment lib path appears twice: $ readelf -d ./bar ... 0x0000000000000001 (NEEDED) Shared library: [libfoo.so] 0x0000000000000001 (NEEDED) Shared library: [ libxerces-c-3.2.so] 0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000000f (RPATH) Library rpath: [/home/vagrant/miniconda3/envs/test-conda/lib:/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib:] ... Which is wrong. Now the build tree binary will pick first any old version of the foo library installed in the environment instead of the version in the build tree. It seems that the Anaconda toolchain is setting the environment lib path into the RPATH by its own. It is also set when installing the binaries, even when CMake is not configured to set the install RPATH: $ make install ... Install the project... -- Install configuration: "" -- Installing: /home/vagrant/miniconda3/envs/test-conda/lib/libfoo.so -- Installing: /home/vagrant/miniconda3/envs/test-conda/include/foo.hpp -- Installing: /home/vagrant/miniconda3/envs/test-conda/bin/bar -- Set runtime path of "/home/vagrant/miniconda3/envs/test-conda/bin/bar" to "" $ bar Hello World! $ readelf -d $CONDA_PREFIX/bin/bar ... 0x0000000000000001 (NEEDED) Shared library: [libfoo.so] 0x0000000000000001 (NEEDED) Shared library: [ libxerces-c-3.2.so] 0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000000f (RPATH) Library rpath: [/home/vagrant/miniconda3/envs/test-conda/lib] ... # deactivate the environment $ conda deactivate -------------------------------------------------- TL;DR I cannot get CMake and the Anaconda compilers and packages working correctly. - On macOS, the Conda environment library path is not added to the build RPATH. - On Linux, the Conda environment library path is always added to the RPATH (in both build and install) independently of CMake. Any advice or workarounds? -- Sebastian Mancilla
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
