On 8/13/2014 12:21 PM, Edwin Wong wrote: > Hi dev-platform, > > TL;DR - Cloud Services and Quality Engineering would like to propose the > creation of a directory named “external" in gecko and gaia repos for > externally dependent tests. > > This enables features married to Cloud Services such as Loop, FindMyDevice, > FirefoxAccounts, and Sync to have centralized tests that can be run locally > or on other continuous integration systems. These tests would live in this > "external” directory along side existing tests (so they live together). These > will be run and sheriffed independently from the main tests. Reviews would > be governed by modules and feature teams. > > More detail: > https://wiki.mozilla.org/QA/External_Tests > > So the goal here is to be able to write tests using our existing test harnesses, but that rely on external services? As a related data point I can point you at the Steeplechase tests[1], which live in-tree but only get run on a specific Steeplechase test cluster.
We have the capability nowadays in Mochitest to designate tests as a 'subsuite'[2], which means they don't get run as part of the normal test run, but only if that subsuite is explicitly run. It would be easy to make a "subsuite = external" set that we could run separately but would still get packaged with the rest of the tests etc. Not to bikeshed you too much, but I'd be in favor of keeping tests near the code they're testing, like we do with our existing tests. We could segregate them off into subdirectories, like "mochitest-external" or "mochitest/external" to make that clear. -Ted 1. http://mxr.mozilla.org/mozilla-central/source/dom/media/tests/mochitest/steeplechase.ini 2. http://mxr.mozilla.org/mozilla-central/source/browser/devtools/debugger/test/browser.ini#3 _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
