On 6/4/14, 1:42 PM, Mike de Boer wrote:
I wasn’t implying that they’re broken at all, it’s just that James was hinting
at that.
Our existing testing frameworks are broken in terms of the goals of the
testharness framework, as far as I understand.
For example, one of the primary goals of testharness is to provide
useful data about which things pass and which things fail even when the
implementation doesn't pass all the tests. This includes things like
uncaught exceptions marking the current test failed and continuing on to
the next logical test, for example. That's because testharness is
designed to be run in an environment where you can assume that the
software being tested will fail some (possibly very large) fraction of
the tests.
This goal is at best a secondary goal for our harnesses, and for
CommonJS, as far as I can tell. We typically expect our software to
pass all of our tests, and do a bit of best-effort to extract useful
information when it fails them, but we don't _really_ do what it would
take to find all the failures. Even ignoring some of our pathological
test suites which don't report anything useful at all on failure,
mochitest-plain will stop a test file (which may contain multiple
logical tests) at the first uncaught exception.
All of which is something James cares about deeply, and something that
absolutely blocks usage of CommonJS in the testharness test suite, but
not the thing Ehsan and I are worrying about right this second.
Again, the implementation of the assertions themselves are still the same as
before, they just live someplace else.
No one has a problem with that, as we've said repeatedly.
But this thread started with, and I quote:
> Now we can say that the ‘old’ XPCShell-test assertion methods are
> deprecated in favour of the Assert.jsm ones.
and
> We’re planning to do the same for Mochitest-browser tests in
> bug 1018226.
which sure sounds like we're planning to deprecate the current assertion
methods in Mochitest-browser. Is that conclusion actually a
misunderstanding of the situation?
I blame this misunderstanding on my apparent inability to explain things well
enough from the get-go. My apologies.
It's not clear to me that there's any misunderstanding here.
-Boris
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
