Also, you mentioned that this is very similar to the lldbinline tests. For that reason, I would actually propose dropping lldbinline tests in favor of this. If they are essentially the same, then it seems better to have 2 types of tests rather than 3, and it seems better for those 2 to be lit + python rather than lldbinline + python, solely for the reason that lit is shared and receives many performance and stability improvements from non LLDB developers.
On Wed, Sep 14, 2016 at 5:13 PM Zachary Turner <ztur...@google.com> wrote: > I'm only saying that we should have an open mind. Obviously there are > (valid!) concerns. If we can't solve them then we can't solve them. The > goal (my goal anyway) is always to make things better, not to use X because > it's X. There's value in consistency, but that doesn't mean that the value > from consistency always outweighs the value you get from doing a custom > thing. > > So what I'm saying is: IF we can find a way to have one test suite across > all of LLVM and it's subprojects, AND it is sufficiently powerful to test > all the things we need to test while remaining maintainable in the long > term, we should absolutely jump on the opportunity. > > But this is one of those things that requires a lot of upfront time > investment before you can actually know if it can work for 100% of things. > Obviously some people don't want to invest their time that way when they're > already satisfied, and I don't blame them. > > But for the people who do, and who think they can solve the problem, > what's the harm? Obviously the burden is on those people to prove that > their vision can be realized. > > But if it is successful, then there's no denying the benefits. 1) Tests > become easier to write. 2) Tests become easier to debug. 3) Consistency > encourages people who have traditionally stayed away from LLDB to > contribute. 4) All the people pouring their effort into the custom thing > can now pour it into the shared thing, so everybody benefits. > > I don't blame you for being scared of command tests. I don't support > their use in the current LLDB test suite either, for exactly the same > reasons you and Jason have expressed. But I do think it's possible to come > up with something that a) doesn't suffer from the same problems, b) allows > testing a ton of extra functionality that is not currently testable through > the api, and c) doesn't rely on python at all. If I'm wrong I'll eat crow > :) > > On Wed, Sep 14, 2016 at 5:00 PM Jim Ingham <jing...@apple.com> wrote: > > Also, w.r.t: > > > Aside from write imperative control flow constructs, which I see as a > positive rather than a negative. > > I wrote a bunch of tests to test that stepping behavior for swift and C > was reasonable. When stepping through source code, there is not one > correct way to write the line tables, and in fact clang & swiftc change how > they describe the source through the line tables all the time. So you have > to do: I stepped, and sometimes I'll get to A, sometimes to B, both are > "right" but I have to do different things in either case. If A, step again > before the next test, if B go to the next test. > > You could "fix" that by only doing one step per test, and taking each of > these as a success. But then you wouldn't test that series of steps don't > accumulate errors, you'd only test "run to a breakpoint and step once." > That would not be good. So your positive would be very much a negative for > this kind of test. > > Traditionally the answer to this has been: we know we have to keep the > current testsuite around but we're just adding other new different ways to > write tests. Now you are saying something very different. Do you really > mean that? > > Jim > >
_______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
