I'm mostly basing this concern on the bad effect this had on gdb all of whose testing was expect based command scraping. gdb is a tool that's much closer to lldb than any of the compiler tools so that experience seems relevant. It's been a decade or so since I worked on gdb, but back when I was working on it, you really had to tread very carefully if you wanted to change the output of, say, the break command, or to thread listings, etc, and a bunch of times I just had to bag some cleanup of output I wanted to do because fixing up all the tests was too time consuming. Because Jason and I had both had this experience when we started working on lldb, we promised ourselves we wouldn't go down this path again...
Jim > On Nov 29, 2017, at 1:43 PM, Davide Italiano <dccitali...@gmail.com> wrote: > > On Wed, Nov 29, 2017 at 1:38 PM, Jim Ingham <jing...@apple.com> wrote: >> I'm a little confused by your response. >> >> My stated objection to command output dependent tests is and has always been >> that they make the test dependent on the details of command output. Over >> time doing so makes it hard to modify command output. This is sort of >> related to interactivity, in the sense that since lldb is an interactive >> tool with lots of different commands producing different reports of >> information we can gather, the desire to improve and modify that output is >> more present than in tools that are less output dependent or whose output is >> meant to be processed, in which case it really is API. Command output for >> lldb is explicitly NOT API, that's why we have a real API for people who >> want to program lldb... So the bad effect of the tests in calcifying this >> output is an issue for lldb where it may not be for other tools. >> > > Hi Jim, > in my experience command output changes can be automated via `sed/grep/awk`. > I'm responsible (and many others are) for fundamental changes in LLVM > tools output (i.e. typeless pointers changed pretty much every > load/store/memory_op* in tree) and I found out changing the output of > tests isn't that troublesome. I'm not yet very familiar with LLDB, so > the story might be different here. > I'm personally in favour of this approach because it scaled very well > in llvm (and lld, FWIW), with many more tests than lldb has. > > -- > Davide _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
