In my experience, many tests were written pre-test-verify, and don't clean up correctly to deal with multiple runs in the same process. They work fine when running as a single session, but blow up in TV. Having skip-if(verify) means that we can at least mark those tests as known broken on TV without taking them out of the test suites completely, especially if we find them in places that we don't know how to fix them and need to file to someone else who may not currently have time to fix.
On Tue, Mar 6, 2018 at 10:49 AM, Emilio Cobos Álvarez <emi...@crisal.io> wrote: > On 03/06/2018 06:04 PM, Geoffrey Brown wrote: > > It is now possible to skip tests in test-verify. Simplify annotate the > > manifest for your test: > > > > [test] > > skip-if = verify > > > > or, for reftests: > > > > skip-if(verify) ... > > > > and the test-verify (TV) test task will not try to verify the annotated > > test. > > > > Please don't abuse this feature! Most TV failures indicate a weakness in > > the test. > > Could you point to an example of a "good" use of this feature? Is it > just to avoid TV failing too intermittently? Is TV unable to run some > tests? When does TV provide no value? > > Thanks! > > -- Emilio > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
