And, btw, this shows the rerun logic working (via the --rerun-all-issues flag):
time test/dotest.py --executable `pwd`/build/Debug/lldb --threads 24 --rerun-all-issues Testing: 416 test suites, 24 threads 377 out of 416 test suites processed - TestSBTypeTypeClass.py Session logs for test failures/errors/unexpected successes will go into directory '2015-12-14-16_44_28' Command invoked: test/dotest.py --executable /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior -p TestMultithreaded.py /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=3:int Configuration: arch=x86_64 compiler=clang ---------------------------------------------------------------------- Collected 8 tests lldb_codesign: no identity found lldb_codesign: no identity found lldb_codesign: no identity found lldb_codesign: no identity found lldb_codesign: no identity found lldb_codesign: no identity found lldb_codesign: no identity found [TestMultithreaded.py FAILED] Command invoked: /usr/bin/python test/dotest.py --executable /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior -p TestMultithreaded.py /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=3:int 396 out of 416 test suites processed - TestMiBreak.py Session logs for test failures/errors/unexpected successes will go into directory '2015-12-14-16_44_28' Command invoked: test/dotest.py --executable /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior -p TestDataFormatterObjC.py /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=12:int Configuration: arch=x86_64 compiler=clang ---------------------------------------------------------------------- Collected 26 tests [TestDataFormatterObjC.py FAILED] Command invoked: /usr/bin/python test/dotest.py --executable /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior -p TestDataFormatterObjC.py /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=12:int 416 out of 416 test suites processed - TestLldbGdbServer.py 2 test files marked for rerun Rerunning the following files: functionalities/data-formatter/data-formatter-objc/TestDataFormatterObjC.py api/multithreaded/TestMultithreaded.py Testing: 2 test suites, 1 thread 2 out of 2 test suites processed - TestMultithreaded.py Test rerun complete ============= Issue Details ============= UNEXPECTED SUCCESS: test_symbol_name_dsym (functionalities/completion/TestCompletion.py) UNEXPECTED SUCCESS: test_symbol_name_dwarf (functionalities/completion/TestCompletion.py) =================== Test Result Summary =================== Test Methods: 1695 Reruns: 30 Success: 1367 Expected Failure: 90 Failure: 0 Error: 0 Exceptional Exit: 0 Unexpected Success: 2 Skip: 236 Timeout: 0 Expected Timeout: 0 On Mon, Dec 14, 2015 at 4:51 PM, Todd Fiala <todd.fi...@gmail.com> wrote: > And that fixed the rest as well. Thanks, Siva! > > -Todd > > On Mon, Dec 14, 2015 at 4:44 PM, Todd Fiala <todd.fi...@gmail.com> wrote: > >> Heh you were skinning the same cat :-) >> >> That fixed the one I was just looking at, running the others now. >> >> On Mon, Dec 14, 2015 at 4:42 PM, Todd Fiala <todd.fi...@gmail.com> wrote: >> >>> Yep, will try now... (I was just looking at the condition testing logic >>> since it looks like something isn't quite right there). >>> >>> On Mon, Dec 14, 2015 at 4:39 PM, Siva Chandra <sivachan...@google.com> >>> wrote: >>> >>>> Can you try again after taking my change at r255584? >>>> >>>> On Mon, Dec 14, 2015 at 4:31 PM, Todd Fiala via lldb-dev >>>> <lldb-dev@lists.llvm.org> wrote: >>>> > I'm having some of these blow up. >>>> > >>>> > In the case of test/lang/c/typedef/Testtypedef.py, it looks like some >>>> of the >>>> > @expected decorators were changed a bit, and perhaps they are not >>>> pound for >>>> > pound the same. For example, this test used to really be marked >>>> XFAIL (via >>>> > an expectedFailureClang directive), but it looks like the current >>>> marking of >>>> > compiler="clang" is either not right or not working, since the test >>>> is run >>>> > on OS X and is treated like it is expected to pass. >>>> > >>>> > I'm drilling into that a bit more, that's just the first of several >>>> that >>>> > fail with these changes on OS X. >>>> > >>>> > On Mon, Dec 14, 2015 at 3:03 PM, Zachary Turner <ztur...@google.com> >>>> wrote: >>>> >> >>>> >> I've checked in r255567 which fixes a problem pointed out by Siva. >>>> It >>>> >> doesn't sound related to in 255542, but looking at those logs I >>>> can't really >>>> >> tell how my CL would be related. If r255567 doesn't fix the bots, >>>> would >>>> >> someone mind helping me briefly? r255542 seems pretty >>>> straightforward, so I >>>> >> don't see why it would have an effect here. >>>> >> >>>> >> On Mon, Dec 14, 2015 at 2:35 PM Todd Fiala <todd.fi...@gmail.com> >>>> wrote: >>>> >>> >>>> >>> Ah yes I see. Thanks, Ying (and Siva! Saw your comments too). >>>> >>> >>>> >>> On Mon, Dec 14, 2015 at 2:34 PM, Ying Chen <chy...@google.com> >>>> wrote: >>>> >>>> >>>> >>>> Seems this is the first build that fails, and it only has one CL >>>> 255542. >>>> >>>> >>>> >>>> >>>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9446 >>>> >>>> I believe Zachary is looking at that problem. >>>> >>>> >>>> >>>> On Mon, Dec 14, 2015 at 2:18 PM, Todd Fiala <todd.fi...@gmail.com> >>>> >>>> wrote: >>>> >>>>> >>>> >>>>> I am seeing several failures on the Ubuntu 14.04 testbot, but >>>> >>>>> unfortunately there are a number of changes that went in at the >>>> same time on >>>> >>>>> that build. The failures I'm seeing are not appearing at all >>>> related to the >>>> >>>>> test running infrastructure. >>>> >>>>> >>>> >>>>> Anybody with a fast Linux system able to take a look to see what >>>> >>>>> exactly is failing there? >>>> >>>>> >>>> >>>>> -Todd >>>> >>>>> >>>> >>>>> On Mon, Dec 14, 2015 at 1:39 PM, Todd Fiala <todd.fi...@gmail.com >>>> > >>>> >>>>> wrote: >>>> >>>>>> >>>> >>>>>> Hi all, >>>> >>>>>> >>>> >>>>>> I just put in the single-worker, low-load, follow-up test run >>>> pass in >>>> >>>>>> r255543. Most of the work for it went in late last week, this >>>> just mostly >>>> >>>>>> flips it on. >>>> >>>>>> >>>> >>>>>> The feature works like this: >>>> >>>>>> >>>> >>>>>> * First test phase works as before: run all tests using whatever >>>> level >>>> >>>>>> of concurrency is normally used. (e.g. 8 works on an >>>> 8-logical-core box). >>>> >>>>>> >>>> >>>>>> * Any timeouts, failures, errors, or anything else that would >>>> have >>>> >>>>>> caused a test failure is eligible for rerun if either (1) it was >>>> marked as a >>>> >>>>>> flakey test via the flakey decorator, or (2) if the >>>> --rerun-all-issues >>>> >>>>>> command line flag is provided. >>>> >>>>>> >>>> >>>>>> * After the first test phase, if there are any tests that met >>>> rerun >>>> >>>>>> eligibility that would have caused a test failure, those get run >>>> using a >>>> >>>>>> serial test phase. Their results will overwrite (i.e. replace) >>>> the previous >>>> >>>>>> result for the given test method. >>>> >>>>>> >>>> >>>>>> The net result should be that tests that were load sensitive and >>>> >>>>>> intermittently fail during the first higher-concurrency test >>>> phase should >>>> >>>>>> (in theory) pass in the second, single worker test phase when >>>> the test suite >>>> >>>>>> is only using a single worker. This should make the test suite >>>> generate >>>> >>>>>> fewer false positives on test failure notification, which should >>>> make >>>> >>>>>> continuous integration servers (testbots) much more useful in >>>> terms of >>>> >>>>>> generating actionable signals caused by version control changes >>>> to the lldb >>>> >>>>>> or related sources. >>>> >>>>>> >>>> >>>>>> Please let me know if you see any issues with this when running >>>> the >>>> >>>>>> test suite using the default output. I'd like to fix this up >>>> ASAP. And for >>>> >>>>>> those interested in the implementation, I'm happy to do >>>> post-commit >>>> >>>>>> review/changes as needed to get it in good shape. >>>> >>>>>> >>>> >>>>>> I'll be watching the builders now and will address any issues >>>> as I >>>> >>>>>> see them. >>>> >>>>>> >>>> >>>>>> Thanks! >>>> >>>>>> -- >>>> >>>>>> -Todd >>>> >>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> -- >>>> >>>>> -Todd >>>> >>>> >>>> >>>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> -- >>>> >>> -Todd >>>> > >>>> > >>>> > >>>> > >>>> > -- >>>> > -Todd >>>> > >>>> > _______________________________________________ >>>> > lldb-dev mailing list >>>> > lldb-dev@lists.llvm.org >>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>>> > >>>> >>> >>> >>> >>> -- >>> -Todd >>> >> >> >> >> -- >> -Todd >> > > > > -- > -Todd > -- -Todd
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
