Hey Ying, Do you mind if we clear the android builder queue to get a build with r255676 in it? There are what looks like at least 3 or 4 builds between now and then, and with timeouts it may take several hours.
-Todd On Tue, Dec 15, 2015 at 1:50 PM, Ying Chen <chy...@google.com> wrote: > Yes, it happens every time for android builder. > > On Tue, Dec 15, 2015 at 1:45 PM, Todd Fiala <todd.fi...@gmail.com> wrote: > >> Hmm, yeah it looks like it did the rerun and then after finishing the >> rerun, it's just hanging. >> >> Let's have a look right after r255676 goes through this builder. I hit a >> hang in the curses output display due to the recursive taking of a lock on >> a lock that was not recursive-enabled. While I would have expected to see >> that with the basic results output that this builder here is using when I >> was testing earlier, it's possible somehow that we're hitting a path here >> that is attempting to recursively take a lock. >> >> Do you know if it is happening every single time a rerun occurs? >> (Hopefully yes?) >> >> -Todd >> >> On Tue, Dec 15, 2015 at 1:38 PM, Todd Fiala <todd.fi...@gmail.com> wrote: >> >>> Yep, I'll have a look! >>> >>> On Tue, Dec 15, 2015 at 12:43 PM, Ying Chen <chy...@google.com> wrote: >>> >>>> Hi Todd, >>>> >>>> It is noticed on lldb android builders that the test_runner didn't exit >>>> after rerun, which caused buildbot timeout since the process was hanging >>>> for over 20 minutes. >>>> Could you please take a look if that's related to your change? >>>> >>>> Please see the following builds. >>>> >>>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-android/builds/4305/steps/test3/logs/stdio >>>> >>>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-android/builds/4305/steps/test7/logs/stdio >>>> >>>> Thanks, >>>> Ying >>>> >>>> On Mon, Dec 14, 2015 at 4:52 PM, Todd Fiala via lldb-dev < >>>> lldb-dev@lists.llvm.org> wrote: >>>> >>>>> And, btw, this shows the rerun logic working (via the >>>>> --rerun-all-issues flag): >>>>> >>>>> time test/dotest.py --executable `pwd`/build/Debug/lldb --threads 24 >>>>> --rerun-all-issues >>>>> Testing: 416 test suites, 24 threads >>>>> 377 out of 416 test suites processed - TestSBTypeTypeClass.py >>>>> >>>>> Session logs for test failures/errors/unexpected successes will go >>>>> into directory '2015-12-14-16_44_28' >>>>> Command invoked: test/dotest.py --executable >>>>> /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 >>>>> --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior >>>>> -p TestMultithreaded.py >>>>> /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test >>>>> --event-add-entries worker_index=3:int >>>>> >>>>> Configuration: arch=x86_64 compiler=clang >>>>> ---------------------------------------------------------------------- >>>>> Collected 8 tests >>>>> >>>>> lldb_codesign: no identity found >>>>> lldb_codesign: no identity found >>>>> lldb_codesign: no identity found >>>>> lldb_codesign: no identity found >>>>> lldb_codesign: no identity found >>>>> lldb_codesign: no identity found >>>>> lldb_codesign: no identity found >>>>> >>>>> [TestMultithreaded.py FAILED] >>>>> Command invoked: /usr/bin/python test/dotest.py --executable >>>>> /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 >>>>> --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior >>>>> -p TestMultithreaded.py >>>>> /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test >>>>> --event-add-entries worker_index=3:int >>>>> 396 out of 416 test suites processed - TestMiBreak.py >>>>> >>>>> Session logs for test failures/errors/unexpected successes will go >>>>> into directory '2015-12-14-16_44_28' >>>>> Command invoked: test/dotest.py --executable >>>>> /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 >>>>> --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior >>>>> -p TestDataFormatterObjC.py >>>>> /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test >>>>> --event-add-entries worker_index=12:int >>>>> >>>>> Configuration: arch=x86_64 compiler=clang >>>>> ---------------------------------------------------------------------- >>>>> Collected 26 tests >>>>> >>>>> >>>>> [TestDataFormatterObjC.py FAILED] >>>>> Command invoked: /usr/bin/python test/dotest.py --executable >>>>> /Users/tfiala/src/lldb-tot/lldb/build/Debug/lldb --threads 24 >>>>> --rerun-all-issues -s 2015-12-14-16_44_28 --results-port 62322 --inferior >>>>> -p TestDataFormatterObjC.py >>>>> /Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test >>>>> --event-add-entries worker_index=12:int >>>>> 416 out of 416 test suites processed - TestLldbGdbServer.py >>>>> 2 test files marked for rerun >>>>> >>>>> >>>>> Rerunning the following files: >>>>> >>>>> functionalities/data-formatter/data-formatter-objc/TestDataFormatterObjC.py >>>>> api/multithreaded/TestMultithreaded.py >>>>> Testing: 2 test suites, 1 thread >>>>> 2 out of 2 test suites processed - TestMultithreaded.py >>>>> Test rerun complete >>>>> >>>>> >>>>> ============= >>>>> Issue Details >>>>> ============= >>>>> UNEXPECTED SUCCESS: test_symbol_name_dsym >>>>> (functionalities/completion/TestCompletion.py) >>>>> UNEXPECTED SUCCESS: test_symbol_name_dwarf >>>>> (functionalities/completion/TestCompletion.py) >>>>> >>>>> =================== >>>>> Test Result Summary >>>>> =================== >>>>> Test Methods: 1695 >>>>> Reruns: 30 >>>>> Success: 1367 >>>>> Expected Failure: 90 >>>>> Failure: 0 >>>>> Error: 0 >>>>> Exceptional Exit: 0 >>>>> Unexpected Success: 2 >>>>> Skip: 236 >>>>> Timeout: 0 >>>>> Expected Timeout: 0 >>>>> >>>>> On Mon, Dec 14, 2015 at 4:51 PM, Todd Fiala <todd.fi...@gmail.com> >>>>> wrote: >>>>> >>>>>> And that fixed the rest as well. Thanks, Siva! >>>>>> >>>>>> -Todd >>>>>> >>>>>> On Mon, Dec 14, 2015 at 4:44 PM, Todd Fiala <todd.fi...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Heh you were skinning the same cat :-) >>>>>>> >>>>>>> That fixed the one I was just looking at, running the others now. >>>>>>> >>>>>>> On Mon, Dec 14, 2015 at 4:42 PM, Todd Fiala <todd.fi...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Yep, will try now... (I was just looking at the condition testing >>>>>>>> logic since it looks like something isn't quite right there). >>>>>>>> >>>>>>>> On Mon, Dec 14, 2015 at 4:39 PM, Siva Chandra < >>>>>>>> sivachan...@google.com> wrote: >>>>>>>> >>>>>>>>> Can you try again after taking my change at r255584? >>>>>>>>> >>>>>>>>> On Mon, Dec 14, 2015 at 4:31 PM, Todd Fiala via lldb-dev >>>>>>>>> <lldb-dev@lists.llvm.org> wrote: >>>>>>>>> > I'm having some of these blow up. >>>>>>>>> > >>>>>>>>> > In the case of test/lang/c/typedef/Testtypedef.py, it looks like >>>>>>>>> some of the >>>>>>>>> > @expected decorators were changed a bit, and perhaps they are >>>>>>>>> not pound for >>>>>>>>> > pound the same. For example, this test used to really be marked >>>>>>>>> XFAIL (via >>>>>>>>> > an expectedFailureClang directive), but it looks like the >>>>>>>>> current marking of >>>>>>>>> > compiler="clang" is either not right or not working, since the >>>>>>>>> test is run >>>>>>>>> > on OS X and is treated like it is expected to pass. >>>>>>>>> > >>>>>>>>> > I'm drilling into that a bit more, that's just the first of >>>>>>>>> several that >>>>>>>>> > fail with these changes on OS X. >>>>>>>>> > >>>>>>>>> > On Mon, Dec 14, 2015 at 3:03 PM, Zachary Turner < >>>>>>>>> ztur...@google.com> wrote: >>>>>>>>> >> >>>>>>>>> >> I've checked in r255567 which fixes a problem pointed out by >>>>>>>>> Siva. It >>>>>>>>> >> doesn't sound related to in 255542, but looking at those logs I >>>>>>>>> can't really >>>>>>>>> >> tell how my CL would be related. If r255567 doesn't fix the >>>>>>>>> bots, would >>>>>>>>> >> someone mind helping me briefly? r255542 seems pretty >>>>>>>>> straightforward, so I >>>>>>>>> >> don't see why it would have an effect here. >>>>>>>>> >> >>>>>>>>> >> On Mon, Dec 14, 2015 at 2:35 PM Todd Fiala < >>>>>>>>> todd.fi...@gmail.com> wrote: >>>>>>>>> >>> >>>>>>>>> >>> Ah yes I see. Thanks, Ying (and Siva! Saw your comments too). >>>>>>>>> >>> >>>>>>>>> >>> On Mon, Dec 14, 2015 at 2:34 PM, Ying Chen <chy...@google.com> >>>>>>>>> wrote: >>>>>>>>> >>>> >>>>>>>>> >>>> Seems this is the first build that fails, and it only has one >>>>>>>>> CL 255542. >>>>>>>>> >>>> >>>>>>>>> >>>> >>>>>>>>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9446 >>>>>>>>> >>>> I believe Zachary is looking at that problem. >>>>>>>>> >>>> >>>>>>>>> >>>> On Mon, Dec 14, 2015 at 2:18 PM, Todd Fiala < >>>>>>>>> todd.fi...@gmail.com> >>>>>>>>> >>>> wrote: >>>>>>>>> >>>>> >>>>>>>>> >>>>> I am seeing several failures on the Ubuntu 14.04 testbot, but >>>>>>>>> >>>>> unfortunately there are a number of changes that went in at >>>>>>>>> the same time on >>>>>>>>> >>>>> that build. The failures I'm seeing are not appearing at >>>>>>>>> all related to the >>>>>>>>> >>>>> test running infrastructure. >>>>>>>>> >>>>> >>>>>>>>> >>>>> Anybody with a fast Linux system able to take a look to see >>>>>>>>> what >>>>>>>>> >>>>> exactly is failing there? >>>>>>>>> >>>>> >>>>>>>>> >>>>> -Todd >>>>>>>>> >>>>> >>>>>>>>> >>>>> On Mon, Dec 14, 2015 at 1:39 PM, Todd Fiala < >>>>>>>>> todd.fi...@gmail.com> >>>>>>>>> >>>>> wrote: >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Hi all, >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> I just put in the single-worker, low-load, follow-up test >>>>>>>>> run pass in >>>>>>>>> >>>>>> r255543. Most of the work for it went in late last week, >>>>>>>>> this just mostly >>>>>>>>> >>>>>> flips it on. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> The feature works like this: >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> * First test phase works as before: run all tests using >>>>>>>>> whatever level >>>>>>>>> >>>>>> of concurrency is normally used. (e.g. 8 works on an >>>>>>>>> 8-logical-core box). >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> * Any timeouts, failures, errors, or anything else that >>>>>>>>> would have >>>>>>>>> >>>>>> caused a test failure is eligible for rerun if either (1) >>>>>>>>> it was marked as a >>>>>>>>> >>>>>> flakey test via the flakey decorator, or (2) if the >>>>>>>>> --rerun-all-issues >>>>>>>>> >>>>>> command line flag is provided. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> * After the first test phase, if there are any tests that >>>>>>>>> met rerun >>>>>>>>> >>>>>> eligibility that would have caused a test failure, those >>>>>>>>> get run using a >>>>>>>>> >>>>>> serial test phase. Their results will overwrite (i.e. >>>>>>>>> replace) the previous >>>>>>>>> >>>>>> result for the given test method. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> The net result should be that tests that were load >>>>>>>>> sensitive and >>>>>>>>> >>>>>> intermittently fail during the first higher-concurrency >>>>>>>>> test phase should >>>>>>>>> >>>>>> (in theory) pass in the second, single worker test phase >>>>>>>>> when the test suite >>>>>>>>> >>>>>> is only using a single worker. This should make the test >>>>>>>>> suite generate >>>>>>>>> >>>>>> fewer false positives on test failure notification, which >>>>>>>>> should make >>>>>>>>> >>>>>> continuous integration servers (testbots) much more useful >>>>>>>>> in terms of >>>>>>>>> >>>>>> generating actionable signals caused by version control >>>>>>>>> changes to the lldb >>>>>>>>> >>>>>> or related sources. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Please let me know if you see any issues with this when >>>>>>>>> running the >>>>>>>>> >>>>>> test suite using the default output. I'd like to fix this >>>>>>>>> up ASAP. And for >>>>>>>>> >>>>>> those interested in the implementation, I'm happy to do >>>>>>>>> post-commit >>>>>>>>> >>>>>> review/changes as needed to get it in good shape. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> I'll be watching the builders now and will address any >>>>>>>>> issues as I >>>>>>>>> >>>>>> see them. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Thanks! >>>>>>>>> >>>>>> -- >>>>>>>>> >>>>>> -Todd >>>>>>>>> >>>>> >>>>>>>>> >>>>> >>>>>>>>> >>>>> >>>>>>>>> >>>>> >>>>>>>>> >>>>> -- >>>>>>>>> >>>>> -Todd >>>>>>>>> >>>> >>>>>>>>> >>>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> -- >>>>>>>>> >>> -Todd >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > -- >>>>>>>>> > -Todd >>>>>>>>> > >>>>>>>>> > _______________________________________________ >>>>>>>>> > lldb-dev mailing list >>>>>>>>> > lldb-dev@lists.llvm.org >>>>>>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>>>>>>>> > >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> -Todd >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -Todd >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -Todd >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> -Todd >>>>> >>>>> _______________________________________________ >>>>> lldb-dev mailing list >>>>> lldb-dev@lists.llvm.org >>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>>>> >>>>> >>>> >>> >>> >>> -- >>> -Todd >>> >> >> >> >> -- >> -Todd >> > > -- -Todd
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
