teemperor created this revision.
teemperor added reviewers: labath, JDevlieghere.
Herald added subscribers: lldb-commits, aprantl.
Herald added a project: LLDB.
teemperor retitled this revision from "[lldb] Fx that a crashing test is marked
as passed as passing/unsupported when it prints UNSUPPORTED before crashing" to
"[lldb] Fix that a crashing test is marked as unsupported when it prints
UNSUPPORTED before crashing".
I added an `abort()` call to some code and noticed that the test suite was
still passing and it just marked my test as "UNSUPPORTED".
It seems the reason for that is that we expect failing tests to print "FAIL:"
which doesn't happen when we crash. If we then also
have an unsupported because we skipped some debug information in the output, we
just mark the test passing because it is unsupported
on the current platform.
This patch marks any test that has a non-zero exit code as failing even if it
doesn't print "FAIL:" (e.g., because it crashed).
Repository:
rLLDB LLDB
https://reviews.llvm.org/D75031
Files:
lldb/test/API/lldbtest.py
Index: lldb/test/API/lldbtest.py
===================================================================
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -114,14 +114,12 @@
return lit.Test.TIMEOUT, output
if exitCode:
- # Match FAIL but not XFAIL.
- for line in out.splitlines() + err.splitlines():
- if line.startswith('FAIL:'):
- return lit.Test.FAIL, output
-
if 'XPASS:' in out or 'XPASS:' in err:
return lit.Test.XPASS, output
+ # Otherwise this is just a failure.
+ return lit.Test.FAIL, output
+
has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
has_passing_tests = 'PASS:' in out or 'PASS:' in err
if has_unsupported_tests and not has_passing_tests:
Index: lldb/test/API/lldbtest.py
===================================================================
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -114,14 +114,12 @@
return lit.Test.TIMEOUT, output
if exitCode:
- # Match FAIL but not XFAIL.
- for line in out.splitlines() + err.splitlines():
- if line.startswith('FAIL:'):
- return lit.Test.FAIL, output
-
if 'XPASS:' in out or 'XPASS:' in err:
return lit.Test.XPASS, output
+ # Otherwise this is just a failure.
+ return lit.Test.FAIL, output
+
has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
has_passing_tests = 'PASS:' in out or 'PASS:' in err
if has_unsupported_tests and not has_passing_tests:
_______________________________________________
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
