This landed as the mach try command. For more details run |./mach help try|
(excerpted below) and see the discussion in bug 1149670. The highlights
here are partly automated try syntax generation for testing changes
relevant to tests in certain directories, and pushing try syntax to the try
server without the benefit of mq.
Chris
"""
Pushes the specified tests to try. The simplest way to specify tests is
by using the -u argument, which will behave as usual for try syntax.
This command also provides a mechanism to select test jobs and tests
within a job by path based on tests present in the tree under that
path. Mochitests, xpcshell tests, and reftests are eligible for
selection by this mechanism. Selected tests will be run in a single
chunk of the relevant suite, at this time in chunk 1.
Specifying platforms is still required with the -p argument (a default
is taken from the AUTOTRY_PLATFORM_HINT environment variable if set).
Tests may be further filtered by passing one or more --tag to the
command.
To run suites in addition to those determined from the tree, they
can be passed to the --extra arguent.
The command requires either its own mercurial extension ("push-to-try",
installable from mach mercurial-setup) or a git repo using git-cinnabar
(available at https://github.com/glandium/git-cinnabar).
Global Arguments:
-v, --verbose Print verbose output.
-l FILENAME, --log-file FILENAME
Filename to write log data to.
--log-interval Prefix log line with interval from last message
rather
than relative time. Note that this is NOT execution
time if there are parallel operations.
--log-no-times Do not prefix log lines with times. By default, mach
will prefix each output line with the time since
command start.
-h, --help Show this help message.
--debug-command Start a Python debugger when command is dispatched.
Command Arguments:
paths Paths to search for tests to run on try.
-n Print detailed information about the resulting test
selection and commands performed.
-p PLATFORMS Platforms to run. (required if not found in the
environment)
-u TESTS Test jobs to run. These will be used in place of
suites
determined by test paths, if any.
--extra EXTRA_TESTS Additional tests to run. These will be added to
suites
determined by test paths, if any.
-b BUILDS Build types to run (d for debug, o for optimized)
--tag TAGS Restrict tests to the given tag (may be specified
multiple times)
--no-push Do not push to try as a result of running this
command
(if specified this command will only print calculated
try syntax and selection info).
"""
On Tue, Mar 31, 2015 at 10:57 AM, Christopher Manchester <
chmanches...@gmail.com> wrote:
> I filed bug 1149670 for the mach try feature.
>
> On Tue, Mar 31, 2015 at 10:39 AM, Andrew Halberstadt <
> ahalberst...@mozilla.com> wrote:
>
>> It's technically already possible by modifying the in-tree mozharness
>> configs here:
>> https://dxr.mozilla.org/mozilla-central/source/testing/config/mozharness
>>
>> However it's not easy to figure out what needs to be modified to get the
>> desired results. Some sort of |mach try| like command is going to be worked
>> on in Q2 to make pushing various test configurations to try easier, I'm not
>> 100% sure how it will be implemented yet though.
>>
>> In the meantime, if you get stuck trying to modify the mozharness
>> configs, let me know and I can come up with an example patch.
>>
>> -Andrew
>>
>>
>> On 31/03/15 01:06 PM, Bobby Holley wrote:
>>
>>> This sounds awesome! Is there an estimate of when we'll be able to use
>>> it for try pushes?
>>>
>>> On Tue, Mar 31, 2015 at 9:30 AM, Andrew Halberstadt <
>>> ahalberst...@mozilla.com <mailto:ahalberst...@mozilla.com>> wrote:
>>>
>>> As of bug 987360, you can now run all tests with a given tag for
>>> mochitest (and variants), xpcshell and marionette based harnesses.
>>> Tags can be applied to either individual tests, or the DEFAULT
>>> section in manifests. Tests can have multiple tags, in which case
>>> they should be comma delimited. To run all tests with a given tag,
>>> pass in --tag <tag name> to the mach command.
>>>
>>> For example, let's say we want to group all mochitest-plain tests
>>> related to canvas together. First we'd add a 'canvas' tag to the
>>> DEFAULT section in
>>> https://dxr.mozilla.org/mozilla-central/source/dom/
>>> canvas/test/mochitest.ini
>>>
>>> [DEFAULT]
>>> tags = canvas
>>>
>>> We notice there is also a canvas related test under dom/media,
>>> namely:
>>> https://dxr.mozilla.org/mozilla-central/source/dom/
>>> media/test/mochitest.ini#541
>>>
>>> Let's pretend it is already tagged with the 'media' tag, but
>>> that's ok, we can add a second tag no problem:
>>>
>>> [test_video_to_canvas.html]
>>> tags = media,canvas
>>>
>>> Repeat above for any other tests or manifests scattered in the
>>> tree that are related to canvas. Now we can run all
>>> mochitest-plain tests with:
>>>
>>> ./mach mochitest-plain --tag canvas
>>>
>>> You can also run the union of two tags by specifying --tag more
>>> than once (though the intersection of two tags is not supported):
>>>
>>> ./mach mochitest-plain --tag canvas --tag media
>>>
>>> So far the xpcshell (./mach xpcshell-test --tag name) and
>>> marionette (./mach marionette-test --tag name) commands are also
>>> supported. Reftest is not supported as it has its own special
>>> manifest format.
>>>
>>> Applying tags to tests will not affect automation or other
>>> people's tags. So each organization or team should feel free to
>>> use tags in whatever creative ways they see fit. Eventually, we'll
>>> start using tags as a foundation for some more advanced features
>>> and analysis. For example, we may implement a way to run all tests
>>> with a given tag across multiple different suites.
>>>
>>> If you have any questions or things aren't working, please let me
>>> know!
>>>
>>> Cheers,
>>> Andrew
>>> _______________________________________________
>>> dev-platform mailing list
>>> dev-platform@lists.mozilla.org <mailto:dev-platform@lists.
>>> mozilla.org>
>>> https://lists.mozilla.org/listinfo/dev-platform
>>>
>>>
>>>
>> _______________________________________________
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>>
>
>
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
