Hi, you may know me as the guy who produces the code coverage results
occasionally: <http://www.tjhsst.edu/~jcranmer/m-ccov/>.
One persistent failure of producing code coverage is the inability to
record code coverage in half of our codebase, the half that is written
in JavaScript. There are several existing JS code coverage solutions,
but all of them would fail to work for a few broad reasons:
1. Mozilla aggressively uses ES6 (and some non-standard) features in its
codebase, and code coverage infrastructure at best lags. Of the half
dozen tools I looked at, only 2 mentioned any support for ES6 (and one
using an ES6-to-ES5 translator, which is unsettling).
2. Importing the coverage database, updating it, and reporting it is
fraught with peril, because we have at least 6 different scopes which
require at least 3 different mechanisms: JS modules, chrome windows,
content windows, workers, JS shell, xpcshell. Some of our code will be
run in multiple scopes
(<https://dxr.mozilla.org/comm-central/source/mozilla/toolkit/components/osfile/osfile.jsm>
can run in both JS modules and chrome workers).
3. Getting a static list of all of our JS code is extremely non-trivial,
since JS can also crop up in non-JS files like XBL files or HTML files.
4. Our scripts sometimes share the same global, sometimes they don't. In
contrast, the target environments of web browsers and node.js always use
one or the other.
If you can't tell from my list of points, I'm highly skeptical that any
instrumentation-based approach will work. However, since we use the same
JS engine for all of our code, if code coverage support is added to
SpiderMonkey, than it is a relatively easy step to add support for JS
code coverage to my periodic code coverage runs. Getting good code
coverage (line and branch coverage) ultimately requires fine-grained
instrumentation (ideally bytecode-level) not presented by the current
Debugger.
I've seen people bring up supporting this sort of stuff in the past,
which usually tends to generate a flurry of "+1 this would be
wonderful!" but ultimately everything peters out before anything gets
done. Some of this may due to be trying to create an overly-general
design that solves all the problems™.
Is there any prospect for this sort of stuff getting done this year?
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
