labath added a comment.
In https://reviews.llvm.org/D46588#1102363, @aprantl wrote:
> > The advantage of the second one is that we will have the ability to
> > inject commands which depend on the results of previous commands (something
> > that I think we will need, sooner or later).
>
> That is worth considering. To write good tests that depend on previous
> results, we probably want to have SBAPI-like Python scripting for lldb-mi
> available. To make that work we would need to introduce an
> lldbmi.HandleCommand("-mycmd...") interface that runs one command, blocks on
> it, and returns the output in a string. And then we would need to export that
> to Python. Since the interface is string-in-string-out, we could probably do
> it via C without even involving SWIG. As an escape hatch to access features
> outside of the mi command set, we could implement the -interpreter-exec mi
> command that passes its input to SBAPI's HandleCommand() if need be.
> Alternatively we could use gtest to write unittest-style lldb-mi tests
> directly in C++, if going via Python is too messy. Again, we would do that in
> a blocking or synchronized I/O mode.
I agree with your analysis. I don't want to give an opinion on the direction,
because:
a) it's not clear to me which way to go, each possibility has different
tradeoffs involved
b) I don't see myself doing anything related to lldb-mi any time soon
What I can offer as an additional data point is the lldb-server test suite.
lldb-server very similar to lldb-mi in a way, as it has an almost-text-based
protocol as its main form of interaction with the world. We also have two ways
of testing it: a python one (`test/testcases/tools/lldb-server` and a gtest one
(`unittests/tools/lldb-server`). the python one is older, but a bit messy
(IMO). The gtest one was started because we had a person interested in
improving lldb-server testing situation. I supported starting a fresh sub-test
suite, because I saw it as a way to solve a couple some fundamental issues in
the old one:
- code duplication: the python test suite needs a complete reimplementation of
the gdb-remote protocol. the c++ test can just hook it to the existing
functionality at an appropriate layer.
- remote testing mess: a very convoluted setup is required to run tests on a
remote (android) device. It's weird that you have to build a host lldb and run
the tests from there, even though the lldb-server and the executables it debugs
run on a different host/arch. The python part of the test needs to run on the
host (there's no easily available python for android), so there's a lot of
flakyness and complexity coming from the port forwarding/network communication).
The c++ based approach could solve these, as the gtest executable could run on
the same device as the lldb-server under test, and it would be possible to run
the tests via check-lldb(-server?) from the cross-build tree instead of copying
things around between two build trees.
Sadly, the person who was supposed to do this left the team. I have tried to
carry this idea on slowly, but haven't yet gotten around to implementing the
most important part (running tests remotely), so I couldn't start migrating the
tests. In retrospect, I am less excited about this idea than I was a year ago,
though I still think it would offer a better testing story for lldb-server.
Overall, I am not sure how much of this relates to lldb-mi (for example, you
probably aren't interested in running it in such a constrained environment),
but I thought it might be interesting for you to look at these before deciding
on a direction. I also don't want to hold up Alexander's internship over this,
as I'm not sure this is what he signed up for. I wanted to bring up the testing
story, because I think there are some issues there, but I'll leave it up to you
(for some definition of "you") to decide on what to do about that.
Repository:
rL LLVM
https://reviews.llvm.org/D46588
_______________________________________________
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
