Yep, Michał also reported this here: https://reviews.llvm.org/D73384
I hope to take a look at that today. On Thu, Jan 30, 2020 at 1:43 PM Vedant Kumar <vedant_ku...@apple.com> wrote: > > > > > On Jan 24, 2020, at 4:18 PM, Jonas Devlieghere via lldb-commits > > <lldb-commits@lists.llvm.org> wrote: > > > > > > Author: Jonas Devlieghere > > Date: 2020-01-24T16:17:55-08:00 > > New Revision: e3a7c7713cd87e37a95a544373cd21f6f06ab94e > > > > URL: > > https://github.com/llvm/llvm-project/commit/e3a7c7713cd87e37a95a544373cd21f6f06ab94e > > DIFF: > > https://github.com/llvm/llvm-project/commit/e3a7c7713cd87e37a95a544373cd21f6f06ab94e.diff > > > > LOG: [lldb/Lit] Change the lldbtest format to behave more like shell test. > > > > The current lldbtest format has a number of shortcomings, all related to > > how we omit information based on why the test fails. For example, a > > successful test would print nothing, even when `-a` is passed to lit. > > It's not up to the test format to decide whether to print something or > > not, that's handled by lit itself. For other test results we would > > sometimes print stdout & stderr, but not always, such as when a timeout > > was reached or we couldn't parse the dotest output. > > > > This patch changes the lldbtest format and makes it behave more like > > lit. We now always print the dotest invocation, the exit code, the > > output to stdout & stderr. If you're used to dealing with ShTests in > > lit, this will feel all very familiar. > > > > Differential revision: https://reviews.llvm.org/D73384 > > > > Added: > > > > > > Modified: > > lldb/test/API/lldbtest.py > > > > Removed: > > > > > > > > ################################################################################ > > diff --git a/lldb/test/API/lldbtest.py b/lldb/test/API/lldbtest.py > > index 349a67f22fb3..864d5ea1df03 100644 > > --- a/lldb/test/API/lldbtest.py > > +++ b/lldb/test/API/lldbtest.py > > @@ -86,33 +86,46 @@ def execute(self, test, litConfig): > > shutil.copy(python, copied_python) > > cmd[0] = copied_python > > > > + timeoutInfo = None > > try: > > out, err, exitCode = lit.util.executeCommand( > > cmd, > > env=test.config.environment, > > timeout=litConfig.maxIndividualTestTime) > > except lit.util.ExecuteCommandTimeoutException: > > - return (lit.Test.TIMEOUT, 'Reached timeout of {} > > seconds'.format( > > - litConfig.maxIndividualTestTime)) > > + timeoutInfo = 'Reached timeout of {} seconds'.format( > > + litConfig.maxIndividualTestTime) > > + > > + output = """Script:\n--\n%s\n--\nExit Code: %d\n""" % ( > > + ' '.join(cmd), exitCode) > > Heads up, when a test times out this prints: > > File "/Users/vsk/src/llvm-project-master/lldb/test/API/lldbtest.py", line > 100, in execute > ' '.join(cmd), exitCode) > UnboundLocalError: local variable 'exitCode' referenced before assignment > > > > + if timeoutInfo is not None: > > + output += """Timeout: %s\n""" % (timeoutInfo,) > > + output += "\n" > > + > > + if out: > > + output += """Command Output (stdout):\n--\n%s\n--\n""" % (out,) > > + if err: > > + output += """Command Output (stderr):\n--\n%s\n--\n""" % (err,) > > + > > + if timeoutInfo: > > + return lit.Test.TIMEOUT, output > > > > if exitCode: > > # Match FAIL but not XFAIL. > > for line in out.splitlines() + err.splitlines(): > > if line.startswith('FAIL:'): > > - return lit.Test.FAIL, out + err > > + return lit.Test.FAIL, output > > > > if 'XPASS:' in out or 'XPASS:' in err: > > - return lit.Test.XPASS, out + err > > + return lit.Test.XPASS, output > > > > has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in > > err > > has_passing_tests = 'PASS:' in out or 'PASS:' in err > > if has_unsupported_tests and not has_passing_tests: > > - return lit.Test.UNSUPPORTED, out + err > > + return lit.Test.UNSUPPORTED, output > > > > passing_test_line = 'RESULT: PASSED' > > if passing_test_line not in out and passing_test_line not in err: > > - msg = ('Unable to find %r in dotest output (exit code > > %d):\n\n%s%s' > > - % (passing_test_line, exitCode, out, err)) > > - return lit.Test.UNRESOLVED, msg > > + return lit.Test.UNRESOLVED, output > > > > - return lit.Test.PASS, '' > > + return lit.Test.PASS, output > > > > > > > > _______________________________________________ > > lldb-commits mailing list > > lldb-commits@lists.llvm.org > > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits > _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
