On Sep 14, 2016, at 5:13 PM, Zachary Turner <ztur...@google.com> wrote:
> I don't blame you for being scared of command tests. I don't support their > use in the current LLDB test suite either, for exactly the same reasons you > and Jason have expressed. But I do think it's possible to come up with > something that a) doesn't suffer from the same problems, b) allows testing a > ton of extra functionality that is not currently testable through the api, > and c) doesn't rely on python at all. If I'm wrong I'll eat crow :) I think your examples are reductive to "printing values is easy to keep consistent" -- but that's only a small part of what a debugger testsuite needs to do. If we want to use lit to drive tests written in terms of SB API, then I'm open to seeing how this tool can be used in lldb. If lit can only be used to write command output tests, then I believe, based on years and years of experience with exactly that kind of test suite in with gdb, that this is a very poor addition to lldb. It will only hamper future lldb development as we want to improve the lldb command line UI - as it did with gdb. Every change to the command line UI breaks hundreds of "easy to write" tests. And who is responsible for cleaning all those up? The people who wrote these easy-to-write command line output matching tests? No, it'll be the person trying to improve the debugger, which is a big disincentive to changing anything else you'll need to wade through the swamp of thousands of accumulated test cases. There's a cost to writing a test case. Either it is paid up front when you write the test in terms of SB API, or it is paid repeatedly over time as people change the UI to the command line tool. I have nothing against lit per se, I'm open to seeing what an SB API based test case in that harness looks like. I have nothing but objections to adding significant new additions to the testsuite that are locked to the command line UI behavior of lldb today. J _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
