JDevlieghere created this revision.
JDevlieghere added reviewers: aprantl, mib.
Herald added a project: All.
JDevlieghere requested review of this revision.
Currently, when an API test has no passing tests but at least one unsupported
test, we report the test as unsupported. This patch slightly changes that
behavior for tests that do not contain any passing tests, but contain both
unsupported tests and expected failures. In that case, I think it's more useful
to report the test as an expected failure rather than being unsupported.
For example, the output below now result in an "expected failure" rather than
"unsupported":
RESULT: PASSED (0 passes, 0 failures, 0 errors, 1 skipped, 3 expected
failures, 0 unexpected successes)
https://reviews.llvm.org/D127258
Files:
lldb/test/API/lldbtest.py
Index: lldb/test/API/lldbtest.py
===================================================================
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -95,8 +95,12 @@
# Otherwise this is just a failure.
return lit.Test.FAIL, output
- has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
has_passing_tests = 'PASS:' in out or 'PASS:' in err
+ has_expected_failures = 'XFAIL:' in out or 'XFAIL:' in err
+ if has_expected_failures and not has_passing_tests:
+ return lit.Test.XFAIL, output
+
+ has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
if has_unsupported_tests and not has_passing_tests:
return lit.Test.UNSUPPORTED, output
Index: lldb/test/API/lldbtest.py
===================================================================
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -95,8 +95,12 @@
# Otherwise this is just a failure.
return lit.Test.FAIL, output
- has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
has_passing_tests = 'PASS:' in out or 'PASS:' in err
+ has_expected_failures = 'XFAIL:' in out or 'XFAIL:' in err
+ if has_expected_failures and not has_passing_tests:
+ return lit.Test.XFAIL, output
+
+ has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
if has_unsupported_tests and not has_passing_tests:
return lit.Test.UNSUPPORTED, output
_______________________________________________
lldb-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
