On Wed, Apr 6, 2011 at 7:39 PM, Dirk Pranke <[email protected]> wrote: > Hi all, > > I am getting increasingly close to having new-run-webkit-tests running > correctly on the apple mac platform with full feature parity to > old-run-webkit-tests. (And, of course, it continues to run on all of > the Chromium bots as well). I am hoping to close the remaining issues > blocking full parity over the next couple weeks. > > You can track the issues where NRWT still falls behind ORWT here: > > https://bugs.webkit.org/show_bug.cgi?id=34984 > > Note that I expect some tests to fail or be flaky under NRWT when it > is running multiple processes/threads at a time. Doing so can both > expose test dependencies on the environment or change the order of > tests that are run by a DumpRenderTree, and I don't consider those to > be bugs that should stop people from using NRWT. The new tool has a > "test_expectations" mechanism that can be used to track expected > failures and we should use it (IMO). > > That said, if there are significant issues making the tool unstable, > causing lots of tests to fail, or just missing functionality, now's > the time to let me know. > > Also, for anyone building / maintaining webkit ports other than the > chromium and apple ones, I strongly encourage you to take another look > at getting NRWT running on your implementations. I hope to at least > make some attempt to run some of the GTK and QT bots myself, but I'm > not about to sign up to test every build variant personally :) > > Note that I believe that NRWT is fully usable today and people should > seriously start using it. That said, I believe the following bugs > should be fixed before we attempt to switch the apple mac port over. > You may wish to wait for these fixes before switching your port over > as well: > > 56731 new-run-webkit-tests doesn't support sample-on-timeout > 56729 new-run-webkit-tests doesn't support WebKit2 > 55907 new-run-webkit-tests should upload crash logs > 37739 new-run-webkit-tests does not report stderr output > 37736 new-run-webkit-tests output is different from (and worse than) > original run-webkit-tests > > There are also a number of issues keeping the apple win port from > working at the moment that I plan to work on shortly. > > There are also a number of bugs currently listed as blocking that I > don't think really qualify. Unless told otherwise, I'm plannning to > remove the blocking flag from the following on Monday 4/11 (if they > haven't been fixed first): > > 57640 [GTK] overlapping drag&drop tests fail on NRWT
Minor revision ... this one might actually qualify as a blocker :) -- Dirk > 55909 new-run-webkit-tests --run-singly option is busted > 55163 new-run-webkit-tests: enable multiple processes by default on Chromium > Win > 47240 new-run-webkit-tests: getting an "error 2" back from ImageDiff > 37426 new-run-webkit-tests should use the ServerProcess abstraction in > chromium.py > 37007 fast/tokenizer/doctype-search-reset.html fails when run out of > order (new-run-webkit-tests) > 35359 fast/repaint/renderer-destruction-by-invalidateSelection-crash.html > fails intermittently > 35266 new-run-webkit-tests --platform=mac-leopard timeout limit should > match run-webkit-tests > 35049 http/tests/security/cross-frame-access-put.html fails > intermittently under new-run-webkit-tests > 35006 fast/dom/global-constructors.html is failing based on previous tests > > Also, just because I don't think they should block a cutover, I do > still think they should be fixed, so don't worry about that :) > > Lastly, over the next couple weeks I also plan to produce some better > documentation for both how to use NRWT and how the code itself is > structured for anyone that wants to hack on it or port it to new > platforms. > > Comments definitely welcome, > > -- Dirk > _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
