I thought that people might be interested in a retrospective on the progress with web-platform-tests in the last year. 2017 has seen big advance in the adoption of web-platform-tests by the wider browser community, and has brought considerable improvements to the associated tooling and infrastructure. So, in no particular order:
== Sync == * About 600 commits from mozilla-central were upstreamed to GitHub. * Google deployed a rapid 2-way sync process between web-platform-tests and the Blink repository. As a result, contributions from Chromium engineers to wpt more than tripled, with over 800 CLs upstreamed. * WebKit and Edge made promising progress on running more tests and making contribution easier. * We developed an improved two-way sync for Firefox which is expected to go live in the new year. This is expected to dramatically improve latency of our synchronisations, and, by tracking upstream changes in bugzilla, improve the visibility of wpt changes to gecko developers. == Test Coverage == * A policy of requiring test changes with normative spec changes was widely adopted by W3C and WHATWG specs. Over half of all browser-relevant specifications now have some variant of this policy. * The CSS test suite was merged into web-platform-tests, bringing almost 30,000 layout tests into the suite. To support running this, we overhauled the wpt reftest runner to significantly improve the performance. * Initial work on "testdriver" landed. This will provide a wpt frontend to test features that aren't available to web content. The initial implementation allows providing user clicks via WebDriver, but there are plans to significantly expand the capabilities in the future. * Significantly improved the support for testing the WebDriver spec. == Infrastructure == * https://wpt.fyi launched. This displays the results of the tests in Firefox, Chrome, Edge and Safari from daily runs of the full suite. There are plans to improve this in the future to make it clearer which test failures are identifying interoperability problems. * wpt-verify job enabled as Tier 2 on treeherder. This is the wpt equivalent of the test-verify job for locating tests that are intermittent as they are initially committed. * Added a CLI in upstream web-platform-tests which provides support for running tests in multiple browsers using a `wpt run <product>` command. This also supports running on Sauce Labs if you need to get a result for a browser that isn't available locally. * Improved the usability of upstream PRs by consolidating all the relevant information into a single dashboard rather than spreading it over multiple GitHub comments. * Improved the upstream CI testing to be faster and cover more browsers. Further improvements to increase reliability and improve the visibility of test results are planned for the near future. Thanks to everyone, both at Mozilla, and in the wider wpt community, who contributed to these changes. I think we've really made a lot of progress on making cross-browser testing in integral part of the process of browser engineering. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
