On Mon, Mar 16, 2015 at 6:03 AM, Andrew Halberstadt < ahalberst...@mozilla.com> wrote:
> On 13/03/15 04:02 PM, Gregory Szorc wrote: > >> Another pain point: running all relevant tests. >> >> Many features have relevant tests across many test suites, sometimes >> spread >> across different directories. When people are hacking on a feature, they >> should have a way to run all relevant tests for that feature. All too >> often >> I've submitted something to Try (or worse landed without Try) only to find >> out I wasn't locally running all tests relevant for the thing that >> changed. >> >> I did some work in bug 987360 around test "tagging." e.g. "mach test fhr" >> would run all tests tagged as relevant to Firefox Health Report. >> > > How does this tagging work? Something that would be quite easy to do would > be to add a "label" attribute to the test manifest. This could be a comma > separated list of arbitrary strings (and could be applied to either > individual tests or entire manifests). E.g > > [test_foo.js] > label=fhr,telemetry > This is pretty much how the proposed patch did it, yes. > ./mach mochitest --label=fhr (or whatever) > That would be one way. I think an argument could be optional. We already have code [1] for resolving arguments to test paths. I think a fallback to tags lookup when the argument doesn't resolve to a filename would make a lot of sense. > IMO there's no need for the build to process this information, and keeping > it in the manifests has better visibility than moz.build. Yes, it should live in test manifests. Test manifests are essentially an extension of moz.build files. They get parsed and incorporated into the moz.build output stream as if the data came from moz.build files. We removed master test manifests (they are now derived from what manifests are included by moz.build files). Test manifests are thus discovered via moz.build files, which requires running moz.build files to find them. Tests also need mozinfo.json to run. This file is produced by config.status, which is generated by configure. Tests and the build system are unfortunately tightly coupled. One thing possibly worth trying is making test manifests definitions execute in moz.build Filesystem Reading Mode [2]. This would allow us to refresh test manifests without having to re-process the build system. Still has some downsides, but it might be a good compromise. [1] https://hg.mozilla.org/mozilla-central/file/436686833af0/python/mozbuild/mozbuild/testing.py#l166 [2] https://gecko.readthedocs.org/en/latest/build/buildsystem/mozbuild-files.html#filesystem-reading-mode _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
