I think this is a great idea, although like others have said, I'd like
to have this implemented inside the test manifests, regardless of
directory structure.
A related piece is reporting; for years, we've had tests like this run
on separate systems, reporting to custom dashboards, because they
weren't appropriate for buildbot, and TBPL can't display data for tests
running anywhere else. With Treeherder, we're now in a position to be
able to display results for tests running anywhere, and this could lead
to greater visibility and adoption of non-buildbot automation.
Such tests would still need to be sheriffed differently (by different
people and/or with different rules), so we'd have to work out what the
views for these tests should be, but using Treeherder to expose test
results for the variety of automation that's run in Mozilla will be a
big win.
Jonathan
On 8/13/2014 9:21 AM, Edwin Wong wrote:
Hi dev-platform,
TL;DR - Cloud Services and Quality Engineering would like to propose the creation of
a directory named “external" in gecko and gaia repos for externally dependent
tests.
This enables features married to Cloud Services such as Loop, FindMyDevice,
FirefoxAccounts, and Sync to have centralized tests that can be run locally or on
other continuous integration systems. These tests would live in this "external”
directory along side existing tests (so they live together). These will be run and
sheriffed independently from the main tests. Reviews would be governed by modules
and feature teams.
More detail:
https://wiki.mozilla.org/QA/External_Tests
Cheers,
Edwin Wong
Cloud Services QE Manager
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
