labath added a reviewer: zturner.
labath added a comment.
Adding Zachary as he's familiar with lit internals.
I'll try to elaborate more on the approach I had in mind. I would split my
approach into a couple of tests.
1. Write a tool which will dump out the list of all tests in the test suite.
This could either be a separate python file, or a special flag to dotest.py,
whichever is easier. It's implementation would basically be like:
for each test subdir:
for each file in subdir:
if file matches Test*.py:
test = __import__(file)
for each class in test:
for each method in class:
if method matches "test*":
print("%s.%s.%s"%(file. class.__name__, method.__name__))
The only tricky part here is the `__import__` line. I think we'll need to do
some fiddling with `sys.path` or whatever to get the test files to import
correctly. We'll probably need to look at how unittest2 imports them to get it
right.
This fiddling is also the reason I think this should be a separate utility.
Since the "test discovery" phase happens in the context of the main lit
process, which is shared with all other test suites, we should try to avoid
messing with it's state too much.
2. have the new test format invoke this utility in its discovery function
(getTestsInDirectory)
3. Invoke something like: `['dotest.py'] + common_args + ['-p', TestFile, '-f',
testclass + '.' + testmethod`]` in the `execute()` function
The thing to resolve here is to figure out how to translate the result of that
command into the test state. Whether we wan't to use just the exit code (this
would give us PASS/FAIL states only), do we want to scrape the command output
to get the rest of the states (XFAIL, SKIP, ...), or do something else..
I am deliberately bypassing lldb-dotest here, as I think it becomes unnecessary
in this setup (so we already remove one layer) -- the common args can be
hardcoded by cmake into `lit.site.cfg` which lit will read for us. And since
now lit is the one executing the test, you don't have to worry about
remembering all the arguments because you can just pass the test name to lit,
and it will figure them out for you (we can still print the full dotest line we
execute to help debugging).
4. Start ripping out all the multiprocessing/driver logic from dotest.
https://reviews.llvm.org/D45215
_______________________________________________
lldb-commits mailing list
[email protected]
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
