Hi everyone,
Mozilla DevTools is exploring implementing parts of the Chrome DevTools
Protocol ("CDP") [0] in Firefox. This is an HTTP, WebSockets, and JSON
based protocol for automating and inspecting running browser pages.
Originally built for the Chrome DevTools, it has seen wider adoption with
outside developers. In addition to Chrome/Chromium, the CDP is supported by
WebKit, Safari, Node.js, and soon Edge, and an ecosystem of libraries and
tools already exists which plug into it, for debugging, extracting
performance data, providing live-preview functionality like the Brackets
editor, and so on. We believe it would be beneficial if these could be
leveraged with Firefox as well.
The initial implementation we have in mind is an alternate target for
third-party integrations to connect to, in addition to the existing Firefox
DevTools Server. The Servo project has also expressed interest in adding
CDP support to improve its own devtools story, and a PR is in flight to
land a CDP server implementation there [1].
I've been working on this project with guidance from Jim Blandy. We've
come up with the following approach:
- A complete, typed Rust implementation of the CDP protocol messages and
(de)serialization lives in the "cdp" crate [2], automatically generated
from the protocol's JSON specification [3] using a build script (this
happens transparently as part of the normal Cargo compilation process).
This comes with Rustdoc API documentation of all messages/types in the
protocol [4] including textual descriptions bundled with the specification
JSON. The cdp crate will likely track the Chrome stable release for which
version of the protocol is supported. A maintainers' script exists which
can find and fetch the appropriate JSON [5].
- The "tokio-cdp" crate [6] builds on the types and (de)serialization
implementation in the cdp crate to provide a server implementation built on
the Tokio asynchronous I/O system. The server side provides traits for
consuming incoming CDP RPC commands, executing them concurrently and
sending back responses, and simultaneously pushing events to the client.
They are generic over the underlying transport, so the same backend
implementation could provide support for "remote" clients plugging in over
HTTP/WebSockets/JSON or, for example, a browser-local client communicating
over IPDL.
- In Servo, a new component plugs into the cdp and tokio-cdp crates and
acts on behalf of connected CDP clients in response to their commands,
communicating with the rest of the Servo constellation. This server is
disabled by default and can be started by passing a "--cdp" flag to the
Servo binary, binding a TCP listener to the loopback interface at the
standard CDP port 9222 (a different port can be specified as an option to
the flag).
- The implementation we envision in Firefox/Gecko would act similarly: a
new Rust component, disabled by default and switched on via a command line
flag, which binds to a local port and mediates between Gecko internals and
clients connected via tokio-cdp.
We chose to build this on Rust and the Tokio event loop, along with the
hyper HTTP library and rust-websocket which plug into Tokio.
Rust and Cargo provide excellent facilities for compile-time code
generation which integrate transparently into the normal build process,
avoiding the need to invoke scripts by hand to keep generated artifacts in
sync. The Rust ecosystem provides libraries such as quote [7] and serde [8]
which allow us to auto-generate an efficient, typed, and self-contained
interface for the entire protocol. This moves the complexity of ingesting,
validating, and extracting information from client messages out of the
Servo- and Gecko-specific backend implementations, helps to ensure they
conform correctly to the protocol specification, and provides a structured
way of upgrading to new protocol versions.
As for Tokio, the event loop and Futures-based model of concurrency it
offers maps well to the Chrome DevTools Protocol. RPC commands typically
execute simultaneously, returning responses in order of completion, while
the server continuously generates events to which the client has
subscribed. Under Tokio we can spawn multiple lightweight Tasks, dispatch
messages to them, and multiplex their responses back over the single client
connection. The Tokio event loop is nicely self-contained to the one or,
optionally, more threads it is allocated, so the rest of the application
doesn't need to be aware of it.
Use of Tokio is becoming a standard in the Rust ecosystem---it's worth
mentioning that Mozilla funds Tokio development [9] and employs some of its
primary developers. Servo currently depends on an older version of the
hyper HTTP client/server library, and consequently this is already present
in the Firefox tree. The current release of hyper is built on top of Tokio,
so upgrading hyper, either as maintenance or to take advantage of the
forthcoming HTTP/2 support, would require pulling in Tokio anyway. The
current release of rust-websocket, from which Servo derives its WebSockets
implementation, also supports Tokio.
The alternative to Tokio for the networking layer would likely be to build
on top of Necko. This presents its own share of problems.
No Rust bindings to Necko currently exist, so writing those in the first
place would become a prerequisite for the rest of the work, and may require
integration with technologies like XPCOM which also lacks Rust support.
Then rewriting the CDP networking layer on top of Necko would duplicate
effort between the CDP implementations for Gecko and Servo, the latter of
which does not presently use Necko and may not be able to take it on as a
dependency.
Furthermore, it is my understanding through conversations with others that
while Necko's HTTP client implementation is mature, what HTTP *server*
implementation exists is bare-bones, and that at present there is no
support for upgrading HTTP connections to WebSockets in order to implement
a WebSocket server. Then building on top of Necko would entail effectively
developing a new library within Necko, when we could instead adopt existing
technologies (Tokio, hyper, rust-websocket) which are already in use by
other Mozilla projects and the surrounding Rust ecosystem.
Feedback, suggestions, and guidance are very much welcome!
https://bugzil.la/1391465 is the bug for the CDP server implementation in
Firefox.
-Michael Smith [:mismith]
[0] https://chromedevtools.github.io/devtools-protocol/
[1] https://github.com/servo/servo/pull/18133
[2] https://github.com/devtools-html/cdp
[3] https://github.com/devtools-html/cdp/blob/master/json/browse
r_protocol.json
[4] https://www.spinda.net/files/mozilla/cdp/doc/cdp/tools/index.html
[5] https://github.com/devtools-html/cdp/blob/master/update_json.sh
[6] https://github.com/devtools-html/tokio-cdp
[7] https://github.com/dtolnay/quote
[8] https://github.com/serde-rs/serde
[9] https://blog.mozilla.org/blog/2017/04/10/mozilla-awards-3650
00-to-open-source-projects-as-part-of-moss/
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
