Do you find that you have time to figure out if rebaselining a test is the right thing to do while you're actively gardening? Maybe I just work too slowly, but I often find that if I'm trying to rebaseline on the fly, it requires that I do at least *some* investigation of the test failure to make sure I'm not rebaselining in an error (or rebaselining a test that is merely flaky) which slows me down enough that I fall behind and inevitably am crushed by the WebKit juggernaut.
-atw On Fri, Jan 8, 2010 at 4:33 PM, Dimitri Glazkov <[email protected]>wrote: > Before heading out for the weekend, I just want to mention this: > rebaseline tool really, really rocks. And yesterday I discovered an > option that I, to my shame, hadn't seen before: -w. This option pulls > baselines from the canary. It's like getting test expectations from > the future! > > In other words, there are no more excuses for you, dear WebKit > gardeners, to commit those ghastly BUG_SOMENAME entries in > test_expectations. > > The workflow is mind-numbingly simple: > > 1) Identify tests that need rebaselining prior to rolling > 2) Add these tests to test_expectations.txt as if you were to commit > them -- except add a REBASELINE flag next to BUG_SOMENAME > 3) Run rebaseline -w > 4) Make sure that the tool ran and removed these entries from > test_expectations.txt > 5) Create CL -- you will notice how new expected result files are > conveniently added for you. > 6) Enjoy regression-free WebKit roll. > > Big thanks to Victor Wang for this amazing instrument of everlasting > harmony. Send him your accolades. Or cash. If you find bugs/quirks, > fix them. > > :DG< > > P.S. As part of gardener/sheriff process overhaul, I will be adding > these and other helpful tips/hints to our gardening doc. > > -- > Chromium Developers mailing list: [email protected] > View archives, change email options, or unsubscribe: > http://groups.google.com/group/chromium-dev >
-- Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
