Hi, This morning tomcat decided to back bug 982842 and a bunch of dependant bugs out for breaking some of the gaia device tests. As I understand things, this is not the first time something like that has happened. However I think that was a mistake, it treated those tests as tier 1 when they pretty clearly do not meet the requirements in https://wiki.mozilla.org/Sheriffing/Job_Visibility_Policy for tier 1 tests. Those tests aren't even on treeherder / tbpl, much less runnable from try. Also https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_unit_tests doesn't document how these tests can be run with an emulator or device. The visibility rules exist in part to make sure that tier 1 tests can be easily reproduced and confirmed to work, however these tests don't come close to being easy to run. The best explanation I've heard for this state of afairs is that people want to get these tests to meet the requirements, but given that we have accepted this excuse for very few tests in the past I don't think that's good enough here. So, until someone gets these tests to be visible on treeherder I don't think we should treat them as a pseudo tier 1 test suite.
Trev
signature.asc
Description: Digital signature
_______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
