Sounds great! I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1161282 for this.
According to https://secure.pub.build.mozilla.org/builddata/reports/reportor/daily/highscores/highscores.html, we still have a ton of people using '-p all -u all' on try On Mon, May 4, 2015 at 5:12 PM, Gregory Szorc <g...@mozilla.com> wrote: > Wait - you're telling me that it is now possible to limit try pushes but > not just jobs but tests within jobs?! Stop the presses: this is huge! If > used by the masses, this could drastically reduce try turnaround times and > decrease automation load and costs. > > Could we encourage use of --tag by having the automation scheduler > up-weight jobs that opt in to reduced load? > > On Thu, Apr 30, 2015 at 4:21 PM, Christopher Manchester < > chmanches...@gmail.com> wrote: > > > You can now add "--tag" arguments to try syntax and they will get passed > to > > test harnesses in your try push. Details of the implementation are in bug > > 978846, but if you're interested in passing other arguments from try > syntax > > to a test harness, this can be done by adding those arguments to > > testing/config/mozharness/try_arguments.py. Note this is still rather > > coarse in the sense that arguments are forwarded without regard for > whether > > a harness supports a particular argument, but I can imagine it being > useful > > in a number of cases (for instance, when testing the feature with > xpcshell > > and "--tag devtools", I was able to get feedback in about ten minutes > > whether things were working rather than waiting for every xpcshell test > to > > run). > > > > Chris > > > > On Thu, Apr 2, 2015 at 2:22 PM, Andrew Halberstadt < > > ahalberst...@mozilla.com > > > wrote: > > > > > Minor update. It was pointed out that other list-like manifestparser > > > attributes (like head and support-files) are whitespace delimited > instead > > > of comma delimited. To be consistent I switched tags to whitespace > > > delimitation as well. > > > > > > E.g both these forms are ok: > > > > > > [test_foo.html] > > > tags = foo bar baz > > > > > > [test_bar.html] > > > tags = > > > foo > > > bar > > > baz > > > > > > -Andrew > > > > > > > > > On 31/03/15 12:30 PM, Andrew Halberstadt wrote: > > > > > >> As of bug 987360, you can now run all tests with a given tag for > > >> mochitest (and variants), xpcshell and marionette based harnesses. > Tags > > >> can be applied to either individual tests, or the DEFAULT section in > > >> manifests. Tests can have multiple tags, in which case they should be > > >> comma delimited. To run all tests with a given tag, pass in --tag <tag > > >> name> to the mach command. > > >> > > >> For example, let's say we want to group all mochitest-plain tests > > >> related to canvas together. First we'd add a 'canvas' tag to the > DEFAULT > > >> section in > > >> > > >> > > > https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/mochitest.ini > > >> > > >> > > >> [DEFAULT] > > >> tags = canvas > > >> > > >> We notice there is also a canvas related test under dom/media, namely: > > >> > > >> > > > https://dxr.mozilla.org/mozilla-central/source/dom/media/test/mochitest.ini#541 > > >> > > >> > > >> Let's pretend it is already tagged with the 'media' tag, but that's > ok, > > >> we can add a second tag no problem: > > >> > > >> [test_video_to_canvas.html] > > >> tags = media,canvas > > >> > > >> Repeat above for any other tests or manifests scattered in the tree > that > > >> are related to canvas. Now we can run all mochitest-plain tests with: > > >> > > >> ./mach mochitest-plain --tag canvas > > >> > > >> You can also run the union of two tags by specifying --tag more than > > >> once (though the intersection of two tags is not supported): > > >> > > >> ./mach mochitest-plain --tag canvas --tag media > > >> > > >> So far the xpcshell (./mach xpcshell-test --tag name) and marionette > > >> (./mach marionette-test --tag name) commands are also supported. > Reftest > > >> is not supported as it has its own special manifest format. > > >> > > >> Applying tags to tests will not affect automation or other people's > > >> tags. So each organization or team should feel free to use tags in > > >> whatever creative ways they see fit. Eventually, we'll start using > tags > > >> as a foundation for some more advanced features and analysis. For > > >> example, we may implement a way to run all tests with a given tag > across > > >> multiple different suites. > > >> > > >> If you have any questions or things aren't working, please let me > know! > > >> > > >> Cheers, > > >> Andrew > > >> > > > > > > _______________________________________________ > > > dev-platform mailing list > > > dev-platform@lists.mozilla.org > > > https://lists.mozilla.org/listinfo/dev-platform > > > > > _______________________________________________ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
