llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-clang Author: Rainer Orth (rorth) <details> <summary>Changes</summary> As discussed in PR #<!-- -->142353, the current testsuite of the `clang` Python bindings has several issues: - It `libclang.so` cannot be loaded into `python` to run the testsuite, the whole `ninja check-all` aborts. - The result of running the testsuite isn't report like the `lit`-based tests, rendering them almost invisible. - The testsuite is disabled in a non-obvious way (`RUN_PYTHON_TESTS`) in `tests/CMakeLists.txt`, which again doesn't show up in the test results. All these issues can be avoided by integrating the Python bindings tests with `lit`, which is what this patch does: - The actual test lives in `clang/test/bindings/python/bindings.sh` and is run by `lit`. - The current `clang/bindings/python/tests` directory (minus the now-subperfluous `CMakeLists.txt`) is moved into the same directory. - The check if `libclang` is loadable (originally from PR #<!-- -->142353) is now handled via a new `lit` feature, `libclang-loadable`. - The various ways to disable the tests have been turned into `XFAIL`s as appropriate. This isn't complete and not completely tested yet. Tested on `sparc-sun-solaris2.11`, `sparcv9-sun-solaris2.11`, `i386-pc-solaris2.11`, `amd64-pc-solaris2.11`, `i686-pc-linux-gnu`, and `x86_64-pc-linux-gnu`. --- Full diff: https://github.com/llvm/llvm-project/pull/142948.diff 38 Files Affected: - (modified) clang/CMakeLists.txt (-1) - (removed) clang/bindings/python/tests/CMakeLists.txt (-66) - (added) clang/test/bindings/python/bindings.sh (+48) - (added) clang/test/bindings/python/lit.local.cfg (+22) - (renamed) clang/test/bindings/python/tests/__init__.py () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/a.inc () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/b.inc () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/compile_commands.json () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/header1.h () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/header2.h () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/header3.h () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/hello.cpp () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/include.cpp () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/parse_arguments.c () - (renamed) clang/test/bindings/python/tests/cindex/INPUTS/testfile.c () - (renamed) clang/test/bindings/python/tests/cindex/__init__.py () - (renamed) clang/test/bindings/python/tests/cindex/test_access_specifiers.py () - (renamed) clang/test/bindings/python/tests/cindex/test_cdb.py () - (renamed) clang/test/bindings/python/tests/cindex/test_code_completion.py () - (renamed) clang/test/bindings/python/tests/cindex/test_comment.py () - (renamed) clang/test/bindings/python/tests/cindex/test_cursor.py () - (renamed) clang/test/bindings/python/tests/cindex/test_cursor_kind.py () - (renamed) clang/test/bindings/python/tests/cindex/test_diagnostics.py () - (renamed) clang/test/bindings/python/tests/cindex/test_enums.py () - (renamed) clang/test/bindings/python/tests/cindex/test_exception_specification_kind.py () - (renamed) clang/test/bindings/python/tests/cindex/test_file.py () - (renamed) clang/test/bindings/python/tests/cindex/test_index.py () - (renamed) clang/test/bindings/python/tests/cindex/test_lib.py () - (renamed) clang/test/bindings/python/tests/cindex/test_linkage.py () - (renamed) clang/test/bindings/python/tests/cindex/test_location.py () - (renamed) clang/test/bindings/python/tests/cindex/test_rewrite.py () - (renamed) clang/test/bindings/python/tests/cindex/test_source_range.py () - (renamed) clang/test/bindings/python/tests/cindex/test_tls_kind.py () - (renamed) clang/test/bindings/python/tests/cindex/test_token_kind.py () - (renamed) clang/test/bindings/python/tests/cindex/test_tokens.py () - (renamed) clang/test/bindings/python/tests/cindex/test_translation_unit.py () - (renamed) clang/test/bindings/python/tests/cindex/test_type.py () - (renamed) clang/test/bindings/python/tests/cindex/util.py () ``````````diff diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt index ab2ac9bc6b9ad..5111953397d04 100644 --- a/clang/CMakeLists.txt +++ b/clang/CMakeLists.txt @@ -533,7 +533,6 @@ if( CLANG_INCLUDE_TESTS ) clang_unit_site_config=${CMAKE_CURRENT_BINARY_DIR}/test/Unit/lit.site.cfg ) add_subdirectory(test) - add_subdirectory(bindings/python/tests) if(CLANG_BUILT_STANDALONE) umbrella_lit_testsuite_end(check-all) diff --git a/clang/bindings/python/tests/CMakeLists.txt b/clang/bindings/python/tests/CMakeLists.txt deleted file mode 100644 index a0ddabc21bb41..0000000000000 --- a/clang/bindings/python/tests/CMakeLists.txt +++ /dev/null @@ -1,66 +0,0 @@ -# Test target to run Python test suite from main build. - -# Avoid configurations including '-include' from interfering with -# our tests by setting CLANG_NO_DEFAULT_CONFIG. -add_custom_target(check-clang-python - COMMAND ${CMAKE_COMMAND} -E env - CLANG_NO_DEFAULT_CONFIG=1 - CLANG_LIBRARY_PATH=$<TARGET_FILE_DIR:libclang> - "${Python3_EXECUTABLE}" -m unittest discover - DEPENDS libclang - WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/..) - -set(RUN_PYTHON_TESTS TRUE) -set_target_properties(check-clang-python PROPERTIES FOLDER "Clang/Tests") - -# Tests require libclang.so which is only built with LLVM_ENABLE_PIC=ON -if(NOT LLVM_ENABLE_PIC) - set(RUN_PYTHON_TESTS FALSE) -endif() - -# Do not try to run if libclang was built with sanitizers because -# the sanitizer library will likely be loaded too late to perform -# interception and will then fail. -# We could use LD_PRELOAD/DYLD_INSERT_LIBRARIES but this isn't -# portable so its easier just to not run the tests when building -# with ASan. -if(NOT LLVM_USE_SANITIZER STREQUAL "") - set(RUN_PYTHON_TESTS FALSE) -endif() - -# Tests fail on Windows, and need someone knowledgeable to fix. -# It's not clear whether it's a test or a valid binding problem. -if(WIN32) - set(RUN_PYTHON_TESTS FALSE) -endif() - -# The Python FFI interface is broken on AIX: https://bugs.python.org/issue38628. -if(${CMAKE_SYSTEM_NAME} MATCHES "AIX") - set(RUN_PYTHON_TESTS FALSE) -endif() - -# AArch64, Hexagon, and Sparc have known test failures that need to be -# addressed. -# SystemZ has broken Python/FFI interface: -# https://reviews.llvm.org/D52840#1265716 -if(${LLVM_NATIVE_ARCH} MATCHES "^(AArch64|Hexagon|Sparc|SystemZ)$") - set(RUN_PYTHON_TESTS FALSE) -endif() - -# Tests will fail if cross-compiling for a different target, as tests will try -# to use the host Python3_EXECUTABLE and make FFI calls to functions in target -# libraries. -if(CMAKE_CROSSCOMPILING) - # FIXME: Consider a solution that allows better control over these tests in - # a crosscompiling scenario. e.g. registering them with lit to allow them to - # be explicitly skipped via appropriate LIT_ARGS, or adding a mechanism to - # allow specifying a python interpreter compiled for the target that could - # be executed using qemu-user. - message(WARNING "check-clang-python not added to check-all as these tests fail in a cross-build setup") - set(RUN_PYTHON_TESTS FALSE) -endif() - -if(RUN_PYTHON_TESTS) - set_property(GLOBAL APPEND PROPERTY - LLVM_ALL_ADDITIONAL_TEST_TARGETS check-clang-python) -endif() diff --git a/clang/test/bindings/python/bindings.sh b/clang/test/bindings/python/bindings.sh new file mode 100755 index 0000000000000..ec4ca55e41822 --- /dev/null +++ b/clang/test/bindings/python/bindings.sh @@ -0,0 +1,48 @@ +#!/bin/sh + +# UNSUPPORTED: !libclang-loadable + +# Tests require libclang.so which is only built with LLVM_ENABLE_PIC=ON +# +# Covered by libclang-loadable, may need to augment test for lack of +# libclang.so. + +# Do not try to run if libclang was built with sanitizers because +# the sanitizer library will likely be loaded too late to perform +# interception and will then fail. +# We could use LD_PRELOAD/DYLD_INSERT_LIBRARIES but this isn't +# portable so its easier just to not run the tests when building +# with ASan. +# +# FIXME: Handle !LLVM_USE_SANITIZER = "". +# lit.site.cfg.py has config.llvm_use_sanitizer = "" + +# Tests fail on Windows, and need someone knowledgeable to fix. +# It's not clear whether it's a test or a valid binding problem. +# XFAIL: target={{.*windows.*}} + +# The Python FFI interface is broken on AIX: https://bugs.python.org/issue38628. +# XFAIL: target={{.*-aix.*}} + +# AArch64, Hexagon, and Sparc have known test failures that need to be +# addressed. +# SystemZ has broken Python/FFI interface: +# https://reviews.llvm.org/D52840#1265716 +# XFAIL: target={{(aarch64|hexagon|sparc*|s390x)-.*}} + +# Tests will fail if cross-compiling for a different target, as tests will try +# to use the host Python3_EXECUTABLE and make FFI calls to functions in target +# libraries. +# +# FIXME: Consider a solution that allows better control over these tests in +# a crosscompiling scenario. e.g. registering them with lit to allow them to +# be explicitly skipped via appropriate LIT_ARGS, or adding a mechanism to +# allow specifying a python interpreter compiled for the target that could +# be executed using qemu-user. +# +# FIXME: Handle CMAKE_CROSSCOMPILING. +# Again, might already be handled by libclang-loadable. + +# RUN: env PYTHONPATH=%S/../../../bindings/python \ +# RUN: CLANG_LIBRARY_PATH=`llvm-config --libdir` \ +# RUN: %python -m unittest discover -s %S/tests diff --git a/clang/test/bindings/python/lit.local.cfg b/clang/test/bindings/python/lit.local.cfg new file mode 100644 index 0000000000000..d3608565f5aef --- /dev/null +++ b/clang/test/bindings/python/lit.local.cfg @@ -0,0 +1,22 @@ +def is_libclang_loadable(): + try: + sys.path.append(os.path.join(config.clang_src_dir, "bindings/python")) + from clang.cindex import Config + conf = Config() + Config.set_library_path(config.clang_lib_dir) + conf.lib + return True + except Exception as e: + # Benign error modes. + if "wrong ELF class: ELFCLASS32" in str(e): + return False + elif "No such file or directory" in str(e): + return False + # Unknown error modes. + else: + return True + +if is_libclang_loadable(): + config.available_features.add("libclang-loadable") + +config.suffixes = ['.sh'] diff --git a/clang/bindings/python/tests/__init__.py b/clang/test/bindings/python/tests/__init__.py similarity index 100% rename from clang/bindings/python/tests/__init__.py rename to clang/test/bindings/python/tests/__init__.py diff --git a/clang/bindings/python/tests/cindex/INPUTS/a.inc b/clang/test/bindings/python/tests/cindex/INPUTS/a.inc similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/a.inc rename to clang/test/bindings/python/tests/cindex/INPUTS/a.inc diff --git a/clang/bindings/python/tests/cindex/INPUTS/b.inc b/clang/test/bindings/python/tests/cindex/INPUTS/b.inc similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/b.inc rename to clang/test/bindings/python/tests/cindex/INPUTS/b.inc diff --git a/clang/bindings/python/tests/cindex/INPUTS/compile_commands.json b/clang/test/bindings/python/tests/cindex/INPUTS/compile_commands.json similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/compile_commands.json rename to clang/test/bindings/python/tests/cindex/INPUTS/compile_commands.json diff --git a/clang/bindings/python/tests/cindex/INPUTS/header1.h b/clang/test/bindings/python/tests/cindex/INPUTS/header1.h similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/header1.h rename to clang/test/bindings/python/tests/cindex/INPUTS/header1.h diff --git a/clang/bindings/python/tests/cindex/INPUTS/header2.h b/clang/test/bindings/python/tests/cindex/INPUTS/header2.h similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/header2.h rename to clang/test/bindings/python/tests/cindex/INPUTS/header2.h diff --git a/clang/bindings/python/tests/cindex/INPUTS/header3.h b/clang/test/bindings/python/tests/cindex/INPUTS/header3.h similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/header3.h rename to clang/test/bindings/python/tests/cindex/INPUTS/header3.h diff --git a/clang/bindings/python/tests/cindex/INPUTS/hello.cpp b/clang/test/bindings/python/tests/cindex/INPUTS/hello.cpp similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/hello.cpp rename to clang/test/bindings/python/tests/cindex/INPUTS/hello.cpp diff --git a/clang/bindings/python/tests/cindex/INPUTS/include.cpp b/clang/test/bindings/python/tests/cindex/INPUTS/include.cpp similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/include.cpp rename to clang/test/bindings/python/tests/cindex/INPUTS/include.cpp diff --git a/clang/bindings/python/tests/cindex/INPUTS/parse_arguments.c b/clang/test/bindings/python/tests/cindex/INPUTS/parse_arguments.c similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/parse_arguments.c rename to clang/test/bindings/python/tests/cindex/INPUTS/parse_arguments.c diff --git a/clang/bindings/python/tests/cindex/INPUTS/testfile.c b/clang/test/bindings/python/tests/cindex/INPUTS/testfile.c similarity index 100% rename from clang/bindings/python/tests/cindex/INPUTS/testfile.c rename to clang/test/bindings/python/tests/cindex/INPUTS/testfile.c diff --git a/clang/bindings/python/tests/cindex/__init__.py b/clang/test/bindings/python/tests/cindex/__init__.py similarity index 100% rename from clang/bindings/python/tests/cindex/__init__.py rename to clang/test/bindings/python/tests/cindex/__init__.py diff --git a/clang/bindings/python/tests/cindex/test_access_specifiers.py b/clang/test/bindings/python/tests/cindex/test_access_specifiers.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_access_specifiers.py rename to clang/test/bindings/python/tests/cindex/test_access_specifiers.py diff --git a/clang/bindings/python/tests/cindex/test_cdb.py b/clang/test/bindings/python/tests/cindex/test_cdb.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_cdb.py rename to clang/test/bindings/python/tests/cindex/test_cdb.py diff --git a/clang/bindings/python/tests/cindex/test_code_completion.py b/clang/test/bindings/python/tests/cindex/test_code_completion.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_code_completion.py rename to clang/test/bindings/python/tests/cindex/test_code_completion.py diff --git a/clang/bindings/python/tests/cindex/test_comment.py b/clang/test/bindings/python/tests/cindex/test_comment.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_comment.py rename to clang/test/bindings/python/tests/cindex/test_comment.py diff --git a/clang/bindings/python/tests/cindex/test_cursor.py b/clang/test/bindings/python/tests/cindex/test_cursor.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_cursor.py rename to clang/test/bindings/python/tests/cindex/test_cursor.py diff --git a/clang/bindings/python/tests/cindex/test_cursor_kind.py b/clang/test/bindings/python/tests/cindex/test_cursor_kind.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_cursor_kind.py rename to clang/test/bindings/python/tests/cindex/test_cursor_kind.py diff --git a/clang/bindings/python/tests/cindex/test_diagnostics.py b/clang/test/bindings/python/tests/cindex/test_diagnostics.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_diagnostics.py rename to clang/test/bindings/python/tests/cindex/test_diagnostics.py diff --git a/clang/bindings/python/tests/cindex/test_enums.py b/clang/test/bindings/python/tests/cindex/test_enums.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_enums.py rename to clang/test/bindings/python/tests/cindex/test_enums.py diff --git a/clang/bindings/python/tests/cindex/test_exception_specification_kind.py b/clang/test/bindings/python/tests/cindex/test_exception_specification_kind.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_exception_specification_kind.py rename to clang/test/bindings/python/tests/cindex/test_exception_specification_kind.py diff --git a/clang/bindings/python/tests/cindex/test_file.py b/clang/test/bindings/python/tests/cindex/test_file.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_file.py rename to clang/test/bindings/python/tests/cindex/test_file.py diff --git a/clang/bindings/python/tests/cindex/test_index.py b/clang/test/bindings/python/tests/cindex/test_index.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_index.py rename to clang/test/bindings/python/tests/cindex/test_index.py diff --git a/clang/bindings/python/tests/cindex/test_lib.py b/clang/test/bindings/python/tests/cindex/test_lib.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_lib.py rename to clang/test/bindings/python/tests/cindex/test_lib.py diff --git a/clang/bindings/python/tests/cindex/test_linkage.py b/clang/test/bindings/python/tests/cindex/test_linkage.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_linkage.py rename to clang/test/bindings/python/tests/cindex/test_linkage.py diff --git a/clang/bindings/python/tests/cindex/test_location.py b/clang/test/bindings/python/tests/cindex/test_location.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_location.py rename to clang/test/bindings/python/tests/cindex/test_location.py diff --git a/clang/bindings/python/tests/cindex/test_rewrite.py b/clang/test/bindings/python/tests/cindex/test_rewrite.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_rewrite.py rename to clang/test/bindings/python/tests/cindex/test_rewrite.py diff --git a/clang/bindings/python/tests/cindex/test_source_range.py b/clang/test/bindings/python/tests/cindex/test_source_range.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_source_range.py rename to clang/test/bindings/python/tests/cindex/test_source_range.py diff --git a/clang/bindings/python/tests/cindex/test_tls_kind.py b/clang/test/bindings/python/tests/cindex/test_tls_kind.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_tls_kind.py rename to clang/test/bindings/python/tests/cindex/test_tls_kind.py diff --git a/clang/bindings/python/tests/cindex/test_token_kind.py b/clang/test/bindings/python/tests/cindex/test_token_kind.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_token_kind.py rename to clang/test/bindings/python/tests/cindex/test_token_kind.py diff --git a/clang/bindings/python/tests/cindex/test_tokens.py b/clang/test/bindings/python/tests/cindex/test_tokens.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_tokens.py rename to clang/test/bindings/python/tests/cindex/test_tokens.py diff --git a/clang/bindings/python/tests/cindex/test_translation_unit.py b/clang/test/bindings/python/tests/cindex/test_translation_unit.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_translation_unit.py rename to clang/test/bindings/python/tests/cindex/test_translation_unit.py diff --git a/clang/bindings/python/tests/cindex/test_type.py b/clang/test/bindings/python/tests/cindex/test_type.py similarity index 100% rename from clang/bindings/python/tests/cindex/test_type.py rename to clang/test/bindings/python/tests/cindex/test_type.py diff --git a/clang/bindings/python/tests/cindex/util.py b/clang/test/bindings/python/tests/cindex/util.py similarity index 100% rename from clang/bindings/python/tests/cindex/util.py rename to clang/test/bindings/python/tests/cindex/util.py `````````` </details> https://github.com/llvm/llvm-project/pull/142948 _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
