On Fri, Feb 10, 2017 at 8:04 PM, James Graham <ja...@hoppipolla.co.uk> wrote: > I don't know what triage-center is, but increasing the visibility of > failures in web-platform-tests is certainly something that I think I should > work on; we are currently letting compat problems slip through > unnecessarily. If there is a preferred way to expose that information to > developers I'm very interested to hear about it.
triage-center[1] is a tool used by Bugzilla component triage owners to identify bugs that haven't been triaged or need action with sections like "Triage decision needed", "Regressions that need a decision for next release", etc. One idea I had is that supposing it's possible to map directories in web-platform-tests to Bugzilla components (even a many-to-many mapping would work, I think), I wonder if we could have a section in that tool that lists "w-p-t failures related to your component that don't have an associated bug" since in a sense those represent untriaged bugs. The workflow for triage owners, would be to file a bug if needed, then update the text expectation file with a reference to the bug (presumably the check-in that updates the expectation file would use the bug number it references and "a=test-only DONTBUILD"?). > As a short term stopgap, you might be interested to know that all upstream > PRs are now run through a stability checker that runs the tests both in > Firefox and Chrome. This includes test submissions that Google upstreams. So > if you look for PRs that you are interested in (e.g. [1] for web-animations) > you can get a heads-up about incoming tests with Firefox/Chrome differences > (work to add Edge and Safari to this stability checker is in progress, but > there are technical difficulties we have to overcome there). Yes, I saw that and was very impressed! That's super useful. For Chrome, however, it would be even more useful if we could run those tests with --enable-experimental-web-platform-features. A lot of the Web Animations features we're testing are implemented in Chrome but disabled unless that flag is set. I imagine that is true of other features too. Thanks again! Brian [1] https://mozilla.github.io/triage-center/ _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
