llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-clang Author: Henrik G. Olsson (hnrklssn) <details> <summary>Changes</summary> This adds a flag to lit for detecting and updating failing tests when possible to do so automatically. The flag uses a plugin architecture where config files can add additional auto-updaters for the types of tests in the test suite. When a test fails with `--update-tests` enabled lit passes the test RUN invocation and output to each registered test updater until one of them signals that it updated the test (or all test updaters have been run). As such it is the responsibility of the test updater to only update tests where it is reasonably certain that it will actually fix the test, or come close to doing so. Initially adds support for UpdateVerifyTests and UpdateTestChecks. The flag is currently only implemented for lit's internal shell, so `--update-tests` implies `LIT_USE_INTERNAL_SHELL=1`. Builds on work in #<!-- -->97369 Fixes #<!-- -->81320 --- Patch is 47.68 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/108425.diff 50 Files Affected: - (modified) clang/test/Sema/lit.local.cfg (+12) - (modified) clang/test/lit.cfg.py (+9) - (added) clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c (+10) - (added) clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected (+10) - (added) clang/test/utils/update-verify-tests/Inputs/infer-indentation.c (+10) - (added) clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected (+13) - (added) clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c (+13) - (added) clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected (+14) - (added) clang/test/utils/update-verify-tests/Inputs/multiple-errors.c (+8) - (added) clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected (+11) - (added) clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c (+10) - (added) clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected (+15) - (added) clang/test/utils/update-verify-tests/Inputs/no-checks.c (+5) - (added) clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected (+6) - (added) clang/test/utils/update-verify-tests/Inputs/no-diags.c (+7) - (added) clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected (+7) - (added) clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c (+6) - (added) clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected (+6) - (added) clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c (+7) - (added) clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected (+7) - (added) clang/test/utils/update-verify-tests/Inputs/update-same-line.c (+6) - (added) clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected (+6) - (added) clang/test/utils/update-verify-tests/Inputs/update-single-check.c (+6) - (added) clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected (+6) - (added) clang/test/utils/update-verify-tests/LitTests/.gitignore (+4) - (added) clang/test/utils/update-verify-tests/LitTests/lit.cfg (+31) - (added) clang/test/utils/update-verify-tests/duplicate-diag.test (+4) - (added) clang/test/utils/update-verify-tests/infer-indentation.test (+3) - (added) clang/test/utils/update-verify-tests/leave-existing-diags.test (+4) - (added) clang/test/utils/update-verify-tests/lit-plugin.test (+5) - (added) clang/test/utils/update-verify-tests/lit.local.cfg (+36) - (added) clang/test/utils/update-verify-tests/multiple-errors.test (+3) - (added) clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test (+3) - (added) clang/test/utils/update-verify-tests/no-checks.test (+3) - (added) clang/test/utils/update-verify-tests/no-diags.test (+4) - (added) clang/test/utils/update-verify-tests/no-expected-diags.test (+4) - (added) clang/test/utils/update-verify-tests/non-default-prefix.test (+4) - (added) clang/test/utils/update-verify-tests/update-same-line.test (+4) - (added) clang/test/utils/update-verify-tests/update-single-check.test (+3) - (added) clang/utils/UpdateVerifyTests/__init__.py (+1) - (added) clang/utils/UpdateVerifyTests/core.py (+452) - (added) clang/utils/UpdateVerifyTests/litplugin.py (+35) - (added) clang/utils/update-verify-tests.py (+41) - (modified) llvm/test/lit.cfg.py (+9) - (modified) llvm/utils/lit/lit/LitConfig.py (+3) - (modified) llvm/utils/lit/lit/TestRunner.py (+12) - (modified) llvm/utils/lit/lit/cl_arguments.py (+6) - (modified) llvm/utils/lit/lit/llvm/config.py (+5) - (modified) llvm/utils/lit/lit/main.py (+1) - (modified) llvm/utils/update_any_test_checks.py (+46-3) ``````````diff diff --git a/clang/test/Sema/lit.local.cfg b/clang/test/Sema/lit.local.cfg index baf1b39ef238cd..b385fd92098a8e 100644 --- a/clang/test/Sema/lit.local.cfg +++ b/clang/test/Sema/lit.local.cfg @@ -2,3 +2,15 @@ config.substitutions = list(config.substitutions) config.substitutions.insert( 0, (r"%clang\b", """*** Do not use the driver in Sema tests. ***""") ) + +if lit_config.update_tests: + import sys + import os + + curdir = os.path.dirname(os.path.realpath(__file__)) + testdir = os.path.dirname(curdir) + clangdir = os.path.dirname(testdir) + utilspath = os.path.join(clangdir, "utils") + sys.path.append(utilspath) + from UpdateVerifyTests.litplugin import verify_test_updater + lit_config.test_updaters.append(verify_test_updater) diff --git a/clang/test/lit.cfg.py b/clang/test/lit.cfg.py index 92a3361ce672e2..c965d3167fe8eb 100644 --- a/clang/test/lit.cfg.py +++ b/clang/test/lit.cfg.py @@ -362,3 +362,12 @@ def calculate_arch_features(arch_string): # possibly be present in system and user configuration files, so disable # default configs for the test runs. config.environment["CLANG_NO_DEFAULT_CONFIG"] = "1" + +if lit_config.update_tests: + import sys + import os + + utilspath = os.path.join(config.llvm_src_root, "utils") + sys.path.append(utilspath) + from update_any_test_checks import utc_lit_plugin + lit_config.test_updaters.append(utc_lit_plugin) diff --git a/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c new file mode 100644 index 00000000000000..d4a92eb4a7874a --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c @@ -0,0 +1,10 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1{{use of undeclared identifier 'a'}} + a = 2; a = 2; + b = 2; b = 2; + // expected-error@+1 3{{use of undeclared identifier 'c'}} + c = 2; c = 2; + // expected-error 2{{asdf}} +} diff --git a/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected new file mode 100644 index 00000000000000..5fd72709e487f5 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/duplicate-diag.c.expected @@ -0,0 +1,10 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1 2{{use of undeclared identifier 'a'}} + a = 2; a = 2; + // expected-error@+1 2{{use of undeclared identifier 'b'}} + b = 2; b = 2; + // expected-error@+1 2{{use of undeclared identifier 'c'}} + c = 2; c = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c new file mode 100644 index 00000000000000..9a58244d92e066 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c @@ -0,0 +1,10 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1 2 {{use of undeclared identifier 'a'}} + a = 2; a = 2; b = 2; b = 2; c = 2; + // expected-error@+1 2 {{asdf}} + d = 2; + e = 2; f = 2; // expected-error 2 {{use of undeclared identifier 'e'}} +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected new file mode 100644 index 00000000000000..127efb003b1883 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/infer-indentation.c.expected @@ -0,0 +1,13 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+3 {{use of undeclared identifier 'c'}} + // expected-error@+2 2 {{use of undeclared identifier 'b'}} + // expected-error@+1 2 {{use of undeclared identifier 'a'}} + a = 2; a = 2; b = 2; b = 2; c = 2; + // expected-error@+1 {{use of undeclared identifier 'd'}} + d = 2; + // expected-error@+1 {{use of undeclared identifier 'f'}} + e = 2; f = 2; // expected-error {{use of undeclared identifier 'e'}} +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c new file mode 100644 index 00000000000000..075d6c80fd4c36 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c @@ -0,0 +1,13 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + a = 2; + // expected-error@-1{{use of undeclared identifier 'a'}} + b = 2;// expected-error{{use of undeclared identifier 'b'}} + c = 2; + // expected-error@7{{use of undeclared identifier 'c'}} + d = 2; // expected-error-re{{use of {{.*}} identifier 'd'}} + + e = 2; // error to trigger mismatch +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected new file mode 100644 index 00000000000000..778ebd4ff4c7b9 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/leave-existing-diags.c.expected @@ -0,0 +1,14 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + a = 2; + // expected-error@-1{{use of undeclared identifier 'a'}} + b = 2;// expected-error{{use of undeclared identifier 'b'}} + c = 2; + // expected-error@7{{use of undeclared identifier 'c'}} + d = 2; // expected-error-re{{use of {{.*}} identifier 'd'}} + + // expected-error@+1{{use of undeclared identifier 'e'}} + e = 2; // error to trigger mismatch +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c new file mode 100644 index 00000000000000..d3fa93ec0e8221 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c @@ -0,0 +1,8 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + a = 2; + b = 2; + + c = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected new file mode 100644 index 00000000000000..aa0cf22c8e8a7b --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/multiple-errors.c.expected @@ -0,0 +1,11 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1{{use of undeclared identifier 'a'}} + a = 2; + // expected-error@+1{{use of undeclared identifier 'b'}} + b = 2; + + // expected-error@+1{{use of undeclared identifier 'c'}} + c = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c new file mode 100644 index 00000000000000..9c5104e693ae57 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c @@ -0,0 +1,10 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + a = 2; b = 2; c = 2; +} + +void bar() { + x = 2; y = 2; z = 2; + // expected-error@-1{{use of undeclared identifier 'x'}} +} diff --git a/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected new file mode 100644 index 00000000000000..01076f76318997 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/multiple-missing-errors-same-line.c.expected @@ -0,0 +1,15 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+3{{use of undeclared identifier 'c'}} + // expected-error@+2{{use of undeclared identifier 'b'}} + // expected-error@+1{{use of undeclared identifier 'a'}} + a = 2; b = 2; c = 2; +} + +void bar() { + x = 2; y = 2; z = 2; + // expected-error@-1{{use of undeclared identifier 'x'}} + // expected-error@-2{{use of undeclared identifier 'y'}} + // expected-error@-3{{use of undeclared identifier 'z'}} +} diff --git a/clang/test/utils/update-verify-tests/Inputs/no-checks.c b/clang/test/utils/update-verify-tests/Inputs/no-checks.c new file mode 100644 index 00000000000000..019698122c3f2b --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/no-checks.c @@ -0,0 +1,5 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + bar = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected b/clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected new file mode 100644 index 00000000000000..42c98d4f77b6ce --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/no-checks.c.expected @@ -0,0 +1,6 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1{{use of undeclared identifier 'bar'}} + bar = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/no-diags.c b/clang/test/utils/update-verify-tests/Inputs/no-diags.c new file mode 100644 index 00000000000000..fddc2ad661fd2d --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/no-diags.c @@ -0,0 +1,7 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1{{asdf}} + int a = 2; +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected b/clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected new file mode 100644 index 00000000000000..acb02badc5655a --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/no-diags.c.expected @@ -0,0 +1,7 @@ +// expected-no-diagnostics +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + int a = 2; +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c new file mode 100644 index 00000000000000..e02b63dcd7a21b --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c @@ -0,0 +1,6 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +// expected-no-diagnostics +void foo() { + a = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected new file mode 100644 index 00000000000000..7fd85f2e064e28 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/no-expected-diags.c.expected @@ -0,0 +1,6 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1{{use of undeclared identifier 'a'}} + a = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c new file mode 100644 index 00000000000000..b7a34e2dfa20d4 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c @@ -0,0 +1,7 @@ +// RUN: %clang_cc1 -verify=check %s +// RUN: diff %s %s.expected +void foo() { + a = 2; // check-error{{asdf}} + // expected-error@-1{ignored}} +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected new file mode 100644 index 00000000000000..087a6be5ea4d2b --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/non-default-prefix.c.expected @@ -0,0 +1,7 @@ +// RUN: %clang_cc1 -verify=check %s +// RUN: diff %s %s.expected +void foo() { + a = 2; // check-error{{use of undeclared identifier 'a'}} + // expected-error@-1{ignored}} +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/update-same-line.c b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c new file mode 100644 index 00000000000000..e448800324e373 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c @@ -0,0 +1,6 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + bar = 2; // expected-error {{asdf}} +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected new file mode 100644 index 00000000000000..ba4387a445fe08 --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/update-same-line.c.expected @@ -0,0 +1,6 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + bar = 2; // expected-error {{use of undeclared identifier 'bar'}} +} + diff --git a/clang/test/utils/update-verify-tests/Inputs/update-single-check.c b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c new file mode 100644 index 00000000000000..e093a8c2ddddbb --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c @@ -0,0 +1,6 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1{{asdf}} + bar = 2; +} diff --git a/clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected new file mode 100644 index 00000000000000..42c98d4f77b6ce --- /dev/null +++ b/clang/test/utils/update-verify-tests/Inputs/update-single-check.c.expected @@ -0,0 +1,6 @@ +// RUN: %clang_cc1 -verify %s +// RUN: diff %s %s.expected +void foo() { + // expected-error@+1{{use of undeclared identifier 'bar'}} + bar = 2; +} diff --git a/clang/test/utils/update-verify-tests/LitTests/.gitignore b/clang/test/utils/update-verify-tests/LitTests/.gitignore new file mode 100644 index 00000000000000..307a6d636e7a28 --- /dev/null +++ b/clang/test/utils/update-verify-tests/LitTests/.gitignore @@ -0,0 +1,4 @@ +*.c +*.c.expected +.lit_test_times.txt +Output diff --git a/clang/test/utils/update-verify-tests/LitTests/lit.cfg b/clang/test/utils/update-verify-tests/LitTests/lit.cfg new file mode 100644 index 00000000000000..dbb22f13ef1303 --- /dev/null +++ b/clang/test/utils/update-verify-tests/LitTests/lit.cfg @@ -0,0 +1,31 @@ +import lit.formats +import lit.llvm +lit.llvm.initialize(lit_config, config) +from lit.llvm import llvm_config +from os.path import dirname + +config.name = "Lit Update Verify Tests" +config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell) +config.suffixes = [ + ".c", +] +config.test_source_root = dirname(__file__) +config.target_triple = None +config.host_triple = None + +config.llvm_tools_dir = lit_config.path[0] # --path is explicitly passed by lit-plugin.test +llvm_config.use_default_substitutions() +llvm_config.use_clang() +config.substitutions.append(("%clang_cc1", "%clang -cc1")) + +if lit_config.update_tests: + import sys + import os + + config.clang_src_dir = dirname(dirname(dirname(dirname(config.test_source_root)))) + utilspath = os.path.join(config.clang_src_dir, "utils") + sys.path.append(utilspath) + from UpdateVerifyTests.litplugin import verify_test_updater + # normally we'd append to the existing list, but when testing + # verify_test_updater we don't want diff_test_updater to accidentally interfere + lit_config.test_updaters = [verify_test_updater] diff --git a/clang/test/utils/update-verify-tests/duplicate-diag.test b/clang/test/utils/update-verify-tests/duplicate-diag.test new file mode 100644 index 00000000000000..3163ce46199c3f --- /dev/null +++ b/clang/test/utils/update-verify-tests/duplicate-diag.test @@ -0,0 +1,4 @@ +# RUN: cp %S/Inputs/duplicate-diag.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests +# RUN: diff -u %S/Inputs/duplicate-diag.c.expected %t.c +# RUN: %clang_cc1 -verify %t.c + diff --git a/clang/test/utils/update-verify-tests/infer-indentation.test b/clang/test/utils/update-verify-tests/infer-indentation.test new file mode 100644 index 00000000000000..6ba2f5d9d505bf --- /dev/null +++ b/clang/test/utils/update-verify-tests/infer-indentation.test @@ -0,0 +1,3 @@ +# RUN: cp %S/Inputs/infer-indentation.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests +# RUN: diff -u %S/Inputs/infer-indentation.c.expected %t.c +# RUN: %clang_cc1 -verify %t.c diff --git a/clang/test/utils/update-verify-tests/leave-existing-diags.test b/clang/test/utils/update-verify-tests/leave-existing-diags.test new file mode 100644 index 00000000000000..cde690ef715a67 --- /dev/null +++ b/clang/test/utils/update-verify-tests/leave-existing-diags.test @@ -0,0 +1,4 @@ +# RUN: cp %S/Inputs/leave-existing-diags.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests +# RUN: diff -u %S/Inputs/leave-existing-diags.c.expected %t.c +# RUN: %clang_cc1 -verify %t.c + diff --git a/clang/test/utils/update-verify-tests/lit-plugin.test b/clang/test/utils/update-verify-tests/lit-plugin.test new file mode 100644 index 00000000000000..f3bbf5ec36bfb5 --- /dev/null +++ b/clang/test/utils/update-verify-tests/lit-plugin.test @@ -0,0 +1,5 @@ +# RUN: cp %S/Inputs/*.c %S/LitTests +# RUN: cp %S/Inputs/*.c.expected %S/LitTests +# RUN: not %{lit} %S/LitTests +# RUN: not %{lit} %S/LitTests --update-tests +# RUN: %{lit} %S/LitTests diff --git a/clang/test/utils/update-verify-tests/lit.local.cfg b/clang/test/utils/update-verify-tests/lit.local.cfg new file mode 100644 index 00000000000000..281c5be37a983a --- /dev/null +++ b/clang/test/utils/update-verify-tests/lit.local.cfg @@ -0,0 +1,36 @@ +import lit.util + +# python 2.7 backwards compatibility +try: + from shlex import quote as shell_quote +except ImportError: + from pipes import quote as shell_quote + +if config.standalone_build: + # These tests require the update-verify-tests.py script from the clang + # source tree, so skip these tests if we are doing standalone builds. + config.unsupported = True +else: + config.suffixes = [".test"] + config.excludes.add("LitTests") + + script_path = os.path.join( + config.clang_src_dir, "utils", "update-verify-tests.py" + ) + python = shell_quote(config.python_executable) + config.substitutions.append( + ( + "%update-verify-tests", + "%s %s" % (python, shell_quote(script_path)), + ) + ) + config.substitutions.append(("%{lit}", "%{lit-no-order-opt} --order=lexical")) + + lit_path = os.path.join(config.llvm_tools_dir, "llvm-lit") + lit_path = os.path.abspath(lit_path) + config.substitutions.append( + ( + "%{lit-no-order-opt}", + f"{python} {lit_path} -j1 --path {config.llvm_tools_dir}" + ) +) diff --git a/clang/test/utils/update-verify-tests/multiple-errors.test b/clang/test/utils/update-verify-tests/multiple-errors.test new file mode 100644 index 00000000000000..1332ef365dc863 --- /dev/null +++ b/clang/test/utils/update-verify-tests/multiple-errors.test @@ -0,0 +1,3 @@ +# RUN: cp %S/Inputs/multiple-errors.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests +# RUN: diff -u %S/Inputs/multiple-errors.c.expected %t.c +# RUN: %clang_cc1 -verify %t.c diff --git a/clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test b/clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test new file mode 100644 index 00000000000000..a9c21cd77e192b --- /dev/null +++ b/clang/test/utils/update-verify-tests/multiple-missing-errors-same-line.test @@ -0,0 +1,3 @@ +# RUN: cp %S/Inputs/multiple-missing-errors-same-line.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests +# RUN: diff -u %S/Inputs/multiple-missing-errors-same-line.c.expected %t.c +# RUN: %clang_cc1 -verify %t.c diff --git a/clang/test/utils/update-verify-tests/no-checks.test b/clang/test/utils/update-verify-tests/no-checks.test new file mode 100644 index 00000000000000..f6ea91fa552be4 --- /dev/null +++ b/clang/test/utils/update-verify-tests/no-checks.test @@ -0,0 +1,3 @@ +# RUN: cp %S/Inputs/no-checks.c %t.c && not %clang_cc1 -verify %t.c 2>&1 | %update-verify-tests +# RUN: diff -u %S/Inputs/no-checks.c.expected %t.c +# RUN: %clang_cc1 -verify %t.c diff --git a/clang/test/utils/update-verify-tests/no-diags.test b/clang/test/utils/update-verify-tests/no-diags.test new file mode 100644 index 00000000000000..464fe8894253b6 --- /dev/null +++ b/clang/test/utils/update-verify-tests/no-diags.test @@ -0,0 +1,4 @@ +# RUN: cp %S/Inputs/no-diag... [truncated] `````````` </details> https://github.com/llvm/llvm-project/pull/108425 _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
