Hi Greg, A few had discussed manifest vs directory separation of tests. I think the directory idea took root because it’s easily identifiable from it’s path, reducing collision with existing tests. I also agree, using directory as a form of meta data is limiting. That said, both concepts can coexist. We should definitely pursue using manifests to define these tests for execution; that can be next.
-edwin On Aug 13, 2014, at 9:39 AM, Gregory Szorc <g...@mozilla.com> wrote: > On 8/13/14 9:21 AM, Edwin Wong wrote: >> Hi dev-platform, >> >> TL;DR - Cloud Services and Quality Engineering would like to propose the >> creation of a directory named “external" in gecko and gaia repos for >> externally dependent tests. >> >> This enables features married to Cloud Services such as Loop, FindMyDevice, >> FirefoxAccounts, and Sync to have centralized tests that can be run locally >> or on other continuous integration systems. These tests would live in this >> "external” directory along side existing tests (so they live together). >> These will be run and sheriffed independently from the main tests. Reviews >> would be governed by modules and feature teams. >> >> More detail: >> https://wiki.mozilla.org/QA/External_Tests > > I'm glad we're going down this road! > > While having a separate directory enforces the separation between tests and > jobs, this separation is partially the result of historical necessity: some > test suites looked at filenames or directory contents for determining what to > execute. > > These days, tests are defined in manifest files. This means that separate > directories don't technically need to exist and the barriers between test > flavors can be less pronounced. > > Have you considered annotating external-accessing tests inside manifests and > having the automation run the subset of tests it is capable of? e.g. |mach > marionette-test --no-external| or |mach test --external-only| > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
