kwk created this revision.
kwk added reviewers: tstellar, phosek.
Herald added subscribers: bzcheeseman, rriddle.
Herald added a project: All.
kwk requested review of this revision.
Herald added subscribers: cfe-commits, stephenneuendorffer.
Herald added a project: clang.
Rationale
=========
This is an opinionated change for the standalone build mode CMake code.
The rationale behind this change is to unify the parts of standalone
builds that historically had to be kept in each and every project. With
the advent of the top-level `cmake` directory inside the LLVM source
tree, it is now possible to bundle the CMake instructions into a file,
aka `cmake/Modules/StandaloneBuildHelpers.cmake`, and include that file
in each project that wants to build in standalone-mode.
Historically the standalone build mode is used mostly by Linux
distributions. Certainly not every LLVM contributor cares about Linux
distributions. To reduce the frictions it makes even more sense to have
a unified place where to keep the specialities of building in standalone
mode.
Affected projects (so far)
--------------------------
This change brings the unified standalone build mode to the clang and
lld project.
Assumptions
===========
One radical assumption for this change is that in order to build clang
or ldd in standalone mode, you have to first build the `llvm` subproject
and *install* it into any location. You can assist the build process to
find LLVM using `find_package(LLVM)` by specifying
`-DCMAKE_PREFIX_PATH=${LLVM_INSTALL_DIR}/lib/cmake/llvm` in the cmake
configuration process.
You have to build the `llvm subproject with utilies included and
installed
(`-DLLVM_INCLUDE_UTILS:BOOL=ON` and `-DLLVM_INSTALL_UTILS:BOOL=ON`. But
I'm sure that this is done most of the time anyways, no?
Don't build as you go: No more cross-project dependencies on LLVM utilties
--------------------------------------------------------------------------
Another assumption is that in standalone build mode it makes no sense to
build clang and try to build an LLVM utility binary like `FileCheck` if
that is missing. This only adds noise to the cmake files and creates an
indirect dependency on the LLVM utilities directory which doesn't exist
in the the clang source tarball. Therefore we go and search for
Don't silently turn off tests
-----------------------------
Before this change, we would silently turn off tests if a binary like
`FileCheck`, `count` or `not` was missing. This is not only dangerous
but IMHO not helpful. If someone asks for tests by passing
`-DLLVM_INCLUDE_TESTS=On` we should error out at configure time because
we cannot fulfil this request when a binary is missing. This is exactly
what this tests does. If you want to check if an LLVM utility binary
exists and what the path to it is, you can call
`get_llvm_utility_binary_path("FileCheck" "FileCheck_EXE")` and it will
get the import location for the target, aka the path to the binary that
was found when we did `find_package(LLVM)`.
NOTE: You can take a look at this small example project which shows you
how importing of an installed project works:
https://github.com/kwk/cmake-export-binary-example/blob/2b429fccef97eb93c1717ceddb27bbe0022339d2/project-b/CMakeLists.txt#L7-L10
Require external LIT in standalone mode
---------------------------------------
We also think that in standalone mode you always want to use an external
lit and not build it as you go. That's why the `find_external_lit` macro
checks if `LLVM_EXTERNAL_LIT` is set and the path exists. If one of
these conditions doesn't hold true, we error out.
TODO:
( ) make sure the correct binaries of `FileCheck` and `count` and `not`
get substituted in lit test files.
( ) get feedback on this change or just opinions
( ) extend usage to other projects like `mlir`, `libomp` and so on.
( ) more encapsulation in `cmake/Modules/StandaloneBuildHelpers.cmake`
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D141050
Files:
clang/CMakeLists.txt
cmake/Modules/StandaloneBuildHelpers.cmake
lld/CMakeLists.txt
Index: lld/CMakeLists.txt
===================================================================
--- lld/CMakeLists.txt
+++ lld/CMakeLists.txt
@@ -23,23 +23,16 @@
# Must go below project(..)
include(GNUInstallDirs)
-if(LLD_BUILT_STANDALONE)
- set(CMAKE_CXX_STANDARD 17 CACHE STRING "C++ standard to conform to")
- set(CMAKE_CXX_STANDARD_REQUIRED YES)
- set(CMAKE_CXX_EXTENSIONS NO)
-
- set(CMAKE_INCLUDE_CURRENT_DIR ON)
-
- find_package(LLVM REQUIRED HINTS "${LLVM_CMAKE_DIR}")
- list(APPEND CMAKE_MODULE_PATH "${LLVM_DIR}")
-
- # Turn into CACHE PATHs for overwritting
- set(LLVM_INCLUDE_DIRS ${LLVM_INCLUDE_DIRS} CACHE PATH "Path to llvm/include and any other header dirs needed")
- set(LLVM_BINARY_DIR "${LLVM_BINARY_DIR}" CACHE PATH "Path to LLVM build tree")
- set(LLVM_MAIN_SRC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../llvm" CACHE PATH "Path to LLVM source tree")
+# Make sure that our source directory is on the current cmake module path so that
+# we can include cmake files from this directory.
+list(INSERT CMAKE_MODULE_PATH 0
+ "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
+ "${LLVM_COMMON_CMAKE_UTILS}/Modules"
+ )
- find_program(LLVM_TABLEGEN_EXE "llvm-tblgen" ${LLVM_TOOLS_BINARY_DIR}
- NO_DEFAULT_PATH)
+if(LLD_BUILT_STANDALONE)
+ include(StandaloneBuildHelpers)
+ get_llvm_utility_binary_path("llvm-tblgen" "LLVM_TABLEGEN_EXE")
# They are used as destination of target generators.
set(LLVM_RUNTIME_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/bin)
@@ -59,53 +52,17 @@
if(LLVM_INCLUDE_TESTS)
find_package(Python3 ${LLVM_MINIMUM_PYTHON_VERSION} REQUIRED
COMPONENTS Interpreter)
-
- # Check prebuilt llvm/utils.
- if(EXISTS ${LLVM_TOOLS_BINARY_DIR}/FileCheck${CMAKE_EXECUTABLE_SUFFIX}
- AND EXISTS ${LLVM_TOOLS_BINARY_DIR}/not${CMAKE_EXECUTABLE_SUFFIX})
- set(LLVM_UTILS_PROVIDED ON)
- endif()
-
- if(EXISTS ${LLVM_MAIN_SRC_DIR}/utils/lit/lit.py)
- # Note: path not really used, except for checking if lit was found
- set(LLVM_LIT ${LLVM_MAIN_SRC_DIR}/utils/lit/lit.py)
- if(NOT LLVM_UTILS_PROVIDED)
- add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/FileCheck utils/FileCheck)
- add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/not utils/not)
- set(LLVM_UTILS_PROVIDED ON)
- set(LLD_TEST_DEPS FileCheck not)
- endif()
- set(UNITTEST_DIR ${LLVM_THIRD_PARTY_DIR}/unittest)
- if(EXISTS ${UNITTEST_DIR}/googletest/include/gtest/gtest.h
- AND NOT EXISTS ${LLVM_LIBRARY_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}gtest${CMAKE_STATIC_LIBRARY_SUFFIX}
- AND EXISTS ${UNITTEST_DIR}/CMakeLists.txt)
- add_subdirectory(${UNITTEST_DIR} third-party/unittest)
- endif()
- else()
- # Seek installed Lit.
- find_program(LLVM_LIT
- NAMES llvm-lit lit.py lit
- PATHS "${LLVM_MAIN_SRC_DIR}/utils/lit"
- DOC "Path to lit.py")
- endif()
-
- if(LLVM_LIT)
- # Define the default arguments to use with 'lit', and an option for the user
- # to override.
- set(LIT_ARGS_DEFAULT "-sv")
- if (MSVC OR XCODE)
- set(LIT_ARGS_DEFAULT "${LIT_ARGS_DEFAULT} --no-progress-bar")
- endif()
- set(LLVM_LIT_ARGS "${LIT_ARGS_DEFAULT}" CACHE STRING "Default options for lit")
-
- get_errc_messages(LLVM_LIT_ERRC_MESSAGES)
-
- # On Win32 hosts, provide an option to specify the path to the GnuWin32 tools.
- if(WIN32 AND NOT CYGWIN)
- set(LLVM_LIT_TOOLS_DIR "" CACHE PATH "Path to GnuWin32 tools")
- endif()
- else()
- set(LLVM_INCLUDE_TESTS OFF)
+ find_external_lit()
+ set_lit_defaults()
+ get_llvm_utility_binary_path("FileCheck" "FileCheck_EXE")
+ get_llvm_utility_binary_path("count" "count_EXE")
+ get_llvm_utility_binary_path("not" "not_EXE")
+
+ set(UNITTEST_DIR ${LLVM_THIRD_PARTY_DIR}/unittest)
+ if(EXISTS ${UNITTEST_DIR}/googletest/include/gtest/gtest.h
+ AND NOT EXISTS ${LLVM_LIBRARY_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}gtest${CMAKE_STATIC_LIBRARY_SUFFIX}
+ AND EXISTS ${UNITTEST_DIR}/CMakeLists.txt)
+ add_subdirectory(${UNITTEST_DIR} third-party/unittest)
endif()
endif()
@@ -153,12 +110,6 @@
"`CMakeFiles'. Please delete them.")
endif()
-# Add path for custom modules.
-list(INSERT CMAKE_MODULE_PATH 0
- "${LLD_SOURCE_DIR}/cmake/modules"
- "${LLVM_COMMON_CMAKE_UTILS}/Modules"
- )
-
include(AddLLD)
option(LLD_USE_VTUNE
Index: cmake/Modules/StandaloneBuildHelpers.cmake
===================================================================
--- /dev/null
+++ cmake/Modules/StandaloneBuildHelpers.cmake
@@ -0,0 +1,109 @@
+# This file provides helper functions and macros for projects that can be built
+# in standalone mode which you often find in linux packaging situations.
+# We also noticed that most functions set a bunch of variables in the same way
+# by default, so we do this while including this file.
+
+include_guard()
+
+# WARNING: The functions in this file only makes sense when building in standalone mode.
+# Therefore it raises a FATAL_ERROR if you're using it in some other
+# context.
+# TODO: Add the XXX_STANDALONE check for anything that is missing here.
+if (NOT (CLANG_BUILT_STANDALONE
+ OR LLD_BUILT_STANDALONE
+ OR COMPILER_RT_STANDALONE_BUILD
+ OR OPENMP_STANDALONE_BUILD
+ OR MLIR_STANDALONE_BUILD
+ OR LLDB_BUILT_STANDALONE))
+ message(FATAL_ERROR "Make sure you build in standalone mode")
+endif()
+
+set(CMAKE_CXX_STANDARD 17 CACHE STRING "C++ standard to conform to")
+set(CMAKE_CXX_STANDARD_REQUIRED YES)
+set(CMAKE_CXX_EXTENSIONS NO)
+
+if(NOT MSVC_IDE)
+set(LLVM_ENABLE_ASSERTIONS ${ENABLE_ASSERTIONS}
+ CACHE BOOL "Enable assertions")
+# Assertions should follow llvm-config's.
+mark_as_advanced(LLVM_ENABLE_ASSERTIONS)
+endif()
+
+# NOTE: The reason to first run `find_package(LLVM)` is that all the imported
+# executables in LLVMExports.cmake are checked for existence. This saves
+# us from checking for them ourselves.
+find_package(LLVM REQUIRED HINTS "${LLVM_CMAKE_DIR}")
+list(APPEND CMAKE_MODULE_PATH "${LLVM_DIR}")
+
+# Turn into CACHE PATHs for overwritting
+set(LLVM_INCLUDE_DIRS ${LLVM_INCLUDE_DIRS} CACHE PATH "Path to llvm/include and any other header dirs needed")
+set(LLVM_BINARY_DIR "${LLVM_BINARY_DIR}" CACHE PATH "Path to LLVM build tree")
+set(LLVM_MAIN_SRC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../llvm" CACHE PATH "Path to LLVM source tree")
+set(LLVM_TOOLS_BINARY_DIR "${LLVM_TOOLS_BINARY_DIR}" CACHE PATH "Path to llvm/bin")
+set(LLVM_LIBRARY_DIR "${LLVM_LIBRARY_DIR}" CACHE PATH "Path to llvm/lib")
+
+# Automatically add the current source and build directories to the include path.
+set(CMAKE_INCLUDE_CURRENT_DIR ON)
+
+# This function tries to get the path to the installed utility binary.
+#
+# - utility: The utility's target name to look up (e.g. FileCheck). If it does
+# not exist, a FATAL_ERROR is raised.
+#
+# - out_var: The variable with this name is set in the caller's scope to hold
+# the path to the utility binary.
+#
+# NOTE: Check the add_llvm_utility macro in AddLLVM.cmake to find out about how
+# utilities are added. They get added to a global list property
+# LLVM_EXPORTS which ends up in a file LLVMExports.cmake that is
+# installed with LLVM. Your utility target must be defined there.
+function(get_llvm_utility_binary_path utility out_var)
+ set(_imploc IMPORTED_LOCATION_NOCONFIG)
+ # Based on the build type that the installed LLVM was built with,
+ # we pick the right import location.
+ if (LLVM_BUILD_TYPE STREQUAL "Release")
+ set(_imploc IMPORTED_LOCATION_RELEASE)
+ elseif (LLVM_BUILD_TYPE STREQUAL "Debug")
+ set(_imploc IMPORTED_LOCATION_DEBUG)
+ elseif (LLVM_BUILD_TYPE STREQUAL "RelWithDebInfo")
+ set(_imploc IMPORTED_LOCATION_RELWITHDEBINFO)
+ elseif (LLVM_BUILD_TYPE STREQUAL "MinSize")
+ set(_imploc IMPORTED_LOCATION_MINSIZE)
+ endif()
+ if (TARGET ${utility})
+ get_target_property(util_path "${utility}" ${_imploc})
+ set(${out_var} "${util_path}" PARENT_SCOPE)
+ else()
+ message(FATAL_ERROR "The utility with the following target name does not exist: \"${utility}\".")
+ endif()
+ unset(_imploc)
+endfunction()
+
+# The set_lit_defaults macro exists because the code existed in multiple
+# locations before.
+macro(set_lit_defaults)
+ set(LIT_ARGS_DEFAULT "-sv")
+ if (MSVC OR XCODE)
+ set(LIT_ARGS_DEFAULT "${LIT_ARGS_DEFAULT} --no-progress-bar")
+ endif()
+ set(LLVM_LIT_ARGS "${LIT_ARGS_DEFAULT}" CACHE STRING "Default options for lit")
+
+ get_errc_messages(LLVM_LIT_ERRC_MESSAGES)
+ set(LLVM_LIT_ERRC_MESSAGES ${LLVM_LIT_ERRC_MESSAGES})
+
+ # On Win32 hosts, provide an option to specify the path to the GnuWin32 tools.
+ if( WIN32 AND NOT CYGWIN )
+ set(LLVM_LIT_TOOLS_DIR "" CACHE PATH "Path to GnuWin32 tools")
+ endif()
+endmacro()
+
+# The find_external_lit macro ensured the variable LLVM_EXTERNAL_LIT is defined
+# and points to an existing file.
+macro(find_external_lit)
+ if (NOT DEFINED LLVM_EXTERNAL_LIT)
+ message(FATAL_ERROR "In standalone build mode you MUST configure with -DLLVM_EXTERNAL_LIT=... pointing to your lit binary")
+ endif()
+ if (NOT EXISTS ${LLVM_EXTERNAL_LIT})
+ message(FATAL_ERROR "Failed to find external lit at place pointed to by LLVM_EXTERNAL_LIT variable: ${LLVM_EXTERNAL_LIT}")
+ endif()
+endmacro()
Index: clang/CMakeLists.txt
===================================================================
--- clang/CMakeLists.txt
+++ clang/CMakeLists.txt
@@ -23,30 +23,16 @@
# Must go below project(..)
include(GNUInstallDirs)
-if(CLANG_BUILT_STANDALONE)
- set(CMAKE_CXX_STANDARD 17 CACHE STRING "C++ standard to conform to")
- set(CMAKE_CXX_STANDARD_REQUIRED YES)
- set(CMAKE_CXX_EXTENSIONS NO)
-
- if(NOT MSVC_IDE)
- set(LLVM_ENABLE_ASSERTIONS ${ENABLE_ASSERTIONS}
- CACHE BOOL "Enable assertions")
- # Assertions should follow llvm-config's.
- mark_as_advanced(LLVM_ENABLE_ASSERTIONS)
- endif()
-
- find_package(LLVM REQUIRED HINTS "${LLVM_CMAKE_DIR}")
- list(APPEND CMAKE_MODULE_PATH "${LLVM_DIR}")
-
- # Turn into CACHE PATHs for overwritting
- set(LLVM_INCLUDE_DIRS ${LLVM_INCLUDE_DIRS} CACHE PATH "Path to llvm/include and any other header dirs needed")
- set(LLVM_BINARY_DIR "${LLVM_BINARY_DIR}" CACHE PATH "Path to LLVM build tree")
- set(LLVM_MAIN_SRC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../llvm" CACHE PATH "Path to LLVM source tree")
- set(LLVM_TOOLS_BINARY_DIR "${LLVM_TOOLS_BINARY_DIR}" CACHE PATH "Path to llvm/bin")
- set(LLVM_LIBRARY_DIR "${LLVM_LIBRARY_DIR}" CACHE PATH "Path to llvm/lib")
+# Make sure that our source directory is on the current cmake module path so that
+# we can include cmake files from this directory.
+list(INSERT CMAKE_MODULE_PATH 0
+ "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
+ "${LLVM_COMMON_CMAKE_UTILS}/Modules"
+ )
- find_program(LLVM_TABLEGEN_EXE "llvm-tblgen" ${LLVM_TOOLS_BINARY_DIR}
- NO_DEFAULT_PATH)
+if(CLANG_BUILT_STANDALONE)
+ include(StandaloneBuildHelpers)
+ get_llvm_utility_binary_path("llvm-tblgen" "LLVM_TABLEGEN_EXE")
# They are used as destination of target generators.
set(LLVM_RUNTIME_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/bin)
@@ -92,70 +78,23 @@
if(LLVM_INCLUDE_TESTS)
find_package(Python3 ${LLVM_MINIMUM_PYTHON_VERSION} REQUIRED
COMPONENTS Interpreter)
-
- # Check prebuilt llvm/utils.
- if(EXISTS ${LLVM_TOOLS_BINARY_DIR}/FileCheck${CMAKE_EXECUTABLE_SUFFIX}
- AND EXISTS ${LLVM_TOOLS_BINARY_DIR}/count${CMAKE_EXECUTABLE_SUFFIX}
- AND EXISTS ${LLVM_TOOLS_BINARY_DIR}/not${CMAKE_EXECUTABLE_SUFFIX})
- set(LLVM_UTILS_PROVIDED ON)
- endif()
-
- # Seek installed Lit.
- find_program(LLVM_LIT
- NAMES llvm-lit lit.py lit
- PATHS "${LLVM_MAIN_SRC_DIR}/utils/lit"
- DOC "Path to lit.py")
-
- if(EXISTS ${LLVM_MAIN_SRC_DIR}/utils/lit/lit.py)
- # Note: path not really used, except for checking if lit was found
- if(EXISTS ${LLVM_MAIN_SRC_DIR}/utils/llvm-lit)
- add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit utils/llvm-lit)
- endif()
- if(NOT LLVM_UTILS_PROVIDED)
- add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/FileCheck utils/FileCheck)
- add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/count utils/count)
- add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/not utils/not)
- set(LLVM_UTILS_PROVIDED ON)
- set(CLANG_TEST_DEPS FileCheck count not)
- endif()
- set(UNITTEST_DIR ${LLVM_THIRD_PARTY_DIR}/unittest)
- if(EXISTS ${UNITTEST_DIR}/googletest/include/gtest/gtest.h
- AND NOT EXISTS ${LLVM_LIBRARY_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}gtest${CMAKE_STATIC_LIBRARY_SUFFIX}
- AND EXISTS ${UNITTEST_DIR}/CMakeLists.txt)
- add_subdirectory(${UNITTEST_DIR} third-party/unittest)
- endif()
- endif()
-
- if(LLVM_LIT)
- # Define the default arguments to use with 'lit', and an option for the user
- # to override.
- set(LIT_ARGS_DEFAULT "-sv")
- if (MSVC OR XCODE)
- set(LIT_ARGS_DEFAULT "${LIT_ARGS_DEFAULT} --no-progress-bar")
- endif()
- set(LLVM_LIT_ARGS "${LIT_ARGS_DEFAULT}" CACHE STRING "Default options for lit")
-
- get_errc_messages(LLVM_LIT_ERRC_MESSAGES)
-
- # On Win32 hosts, provide an option to specify the path to the GnuWin32 tools.
- if( WIN32 AND NOT CYGWIN )
- set(LLVM_LIT_TOOLS_DIR "" CACHE PATH "Path to GnuWin32 tools")
- endif()
- else()
- set(LLVM_INCLUDE_TESTS OFF)
+ find_external_lit()
+ set_lit_defaults()
+ get_llvm_utility_binary_path("FileCheck" "FileCheck_EXE")
+ get_llvm_utility_binary_path("count" "count_EXE")
+ get_llvm_utility_binary_path("not" "not_EXE")
+
+ set(UNITTEST_DIR ${LLVM_THIRD_PARTY_DIR}/unittest)
+ if(EXISTS ${UNITTEST_DIR}/googletest/include/gtest/gtest.h
+ AND NOT EXISTS ${LLVM_LIBRARY_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}gtest${CMAKE_STATIC_LIBRARY_SUFFIX}
+ AND EXISTS ${UNITTEST_DIR}/CMakeLists.txt)
+ add_subdirectory(${UNITTEST_DIR} third-party/unittest)
endif()
umbrella_lit_testsuite_begin(check-all)
endif() # LLVM_INCLUDE_TESTS
endif() # standalone
-# Make sure that our source directory is on the current cmake module path so that
-# we can include cmake files from this directory.
-list(INSERT CMAKE_MODULE_PATH 0
- "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
- "${LLVM_COMMON_CMAKE_UTILS}/Modules"
- )
-
# This allows disabling clang's XML dependency even if LLVM finds libxml2.
# By default, clang depends on libxml2 if LLVM does.
option(CLANG_ENABLE_LIBXML2 "Whether libclang may depend on libxml2"
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
