Looks like the browser_tests launcher needs to be updated, because its exit code was 1 (that's why the bot went red).
On Fri, Dec 18, 2009 at 04:33, Lei Zhang <[email protected]> wrote: > Does FLAKY_ work on all tests? The test run [1] for my most recent > check-in turned a bot red, even though the only test that failed is > one marked FLAKY_. > > [red] browser_tests [browser_tests 896 flaky did not complete crashed > or hung] [197 seconds] [/red] > > and in the log; [2] > > ... > Note: Google Test filter = AutocompleteBrowserTest.YOU HAVE 8 FLAKY TESTS > [==========] Running 0 tests from 0 test cases. > [==========] 0 tests from 0 test cases ran. (0 ms total) > [ PASSED ] 0 tests. > YOU HAVE 8 FLAKY TESTS > > 113 tests run > 1 test failed > Failing tests: > ExtensionBrowserTest.FLAKY_AutoUpdate > program finished with exit code 1 > elapsedTime=197.683425 > > > [1] > http://build.chromium.org/buildbot/waterfall/builders/Linux%20Builder%20(Views%20dbg)/builds/5784 > [2] > http://build.chromium.org/buildbot/waterfall/builders/Linux%20Builder%20(Views%20dbg)/builds/5784/steps/browser_tests/logs/stdio > > On Tue, Oct 6, 2009 at 4:02 PM, Nicolas Sylvain <[email protected]> wrote: >> Hello, >> We currently have more than 50 unit tests that are disabled. Most of them >> because they were flaky. >> Disabling tests is bad because we lose complete coverage on them, so I >> implemented a way to mark >> tests as "flaky". >> The same way you disable a test with DISABLED_ at the beginning of its name, >> you can now mark >> is as flaky with FLAKY_. The behavior is exactly the same as any other >> running tests. You will still >> be able to see when it fails (and why). The only difference is that if only >> FLAKY_ tests failed, the >> buildbot/trybots won't consider it as a failure. On the waterfall, it will >> show the box as orange with the >> list of all flaky tests that failed (pending one more buildbot restart). On >> the console view it will stay >> green. >> But.. this is not a toy. Flaky tests are bad. We should mark tests flaky >> only if we really have to, and >> if you do, please make sure to file a P1 bug. Set the owner of the bug to >> whoever regressed the test. >> If you can't find who regressed the test, assign it to the person who >> originally wrote the test. >> Once we start tagging the flaky tests, we will monitor the flakiness >> dashboard and make sure >> that a test that is no longer flaky has its FLAKY_ tag removed. >> Let me know if you have questions. >> Thanks >> Nicolas >> >> --~--~---------~--~----~------------~-------~--~----~ >> Chromium Developers mailing list: [email protected] >> View archives, change email options, or unsubscribe: >> http://groups.google.com/group/chromium-dev >> >> -~----------~----~----~----~------~----~------~--~--- >> >> > > -- > Chromium Developers mailing list: [email protected] > View archives, change email options, or unsubscribe: > http://groups.google.com/group/chromium-dev > -- Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
