It'd be easy to update FileCheck tests when changing the debugger (this happens all the time in clang/swift). OTOH, the verbosity of the python API means that fewer tests get written. I see a real need to make expressive tests easier to write.
vedant > On Aug 14, 2018, at 5:38 PM, Jason Molenda <jmole...@apple.com> wrote: > > I'd argue against this approach because it's exactly why the lit tests don't > run against the lldb driver -- they're hardcoding the output of the lldb > driver command into the testsuite and these will eventually make it much more > difficult to change and improve the driver as we've accumulated this style of > test. > > This is a perfect test for a normal SB API. Run to your breakpoints and > check the stack frames. > > f0 = thread.GetFrameAtIndex(0) > check that f0.GetFunctionName() == sink > check that f0.IsArtifical() == True > check that f0.GetLineEntry().GetLine() == expected line number > > > it's more verbose, but it's also much more explicit about what it's checking, > and easy to see what has changed if there is a failure. > > > J > >> On Aug 14, 2018, at 5:31 PM, Vedant Kumar via lldb-dev >> <lldb-dev@lists.llvm.org> wrote: >> >> Hello, >> >> I'd like to make FileCheck available within lldb inline tests, in addition >> to existing helpers like 'runCmd' and 'expect'. >> >> My motivation is that several tests I'm working on can't be made as rigorous >> as they need to be without FileCheck-style checks. In particular, the >> 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't allow >> me to verify the ordering of checked input, to be stringent about line >> numbers, or to capture & reuse snippets of text from the input stream. >> >> I'd curious to know if anyone else is interested or would be willing to >> review this (https://reviews.llvm.org/D50751). >> >> Here's an example of an inline test which benefits from FileCheck-style >> checking. This test is trying to check that certain frames appear in a >> backtrace when stopped inside of the "sink" function. Notice that without >> FileCheck, it's not possible to verify the order in which frames are >> printed, and that dealing with line numbers would be cumbersome. >> >> ``` >> --- >> a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp >> +++ >> b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp >> @@ -9,16 +9,21 @@ >> >> volatile int x; >> >> +// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt] >> void __attribute__((noinline)) sink() { >> - x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3', >> 'sink']) >> + x++; //% self.filecheck("bt", "main.cpp") >> } >> >> +// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial] >> void __attribute__((noinline)) func3() { sink(); /* tail */ } >> >> +// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt] >> void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /* >> regular */ } >> >> +// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial] >> void __attribute__((noinline)) func1() { func2(); /* tail */ } >> >> +// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt] >> int __attribute__((disable_tail_calls)) main() { >> func1(); /* regular */ >> return 0; >> ``` >> >> For reference, here's the output of the "bt" command: >> >> ``` >> runCmd: bt >> output: * thread #1, queue = 'com.apple.main-thread', stop reason = >> breakpoint 1.1 >> * frame #0: 0x000000010c6a6f64 a.out`sink() at main.cpp:14 [opt] >> frame #1: 0x000000010c6a6f70 a.out`func3() at main.cpp:15 [opt] >> [artificial] >> frame #2: 0x000000010c6a6f89 a.out`func2() at main.cpp:21 [opt] >> frame #3: 0x000000010c6a6f90 a.out`func1() at main.cpp:21 [opt] >> [artificial] >> frame #4: 0x000000010c6a6fa9 a.out`main at main.cpp:28 [opt] >> ``` >> >> thanks, >> vedant >> _______________________________________________ >> lldb-dev mailing list >> lldb-dev@lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
