On Wed, Aug 30, 2017 at 3:55 PM, Michael Smith <li...@spinda.net> wrote:
> Hi everyone,
>
> Mozilla DevTools is exploring implementing parts of the Chrome DevTools
> Protocol ("CDP") [0] in Firefox. This is an HTTP, WebSockets, and JSON
> based protocol for automating and inspecting running browser pages.
>
> Originally built for the Chrome DevTools, it has seen wider adoption with
> outside developers. In addition to Chrome/Chromium, the CDP is supported by
> WebKit, Safari, Node.js, and soon Edge, and an ecosystem of libraries and
> tools already exists which plug into it, for debugging, extracting
> performance data, providing live-preview functionality like the Brackets
> editor, and so on. We believe it would be beneficial if these could be
> leveraged with Firefox as well.
>
> The initial implementation we have in mind is an alternate target for
> third-party integrations to connect to, in addition to the existing Firefox
> DevTools Server. The Servo project has also expressed interest in adding
> CDP support to improve its own devtools story, and a PR is in flight to
> land a CDP server implementation there [1].
>
> I've been working on this project with guidance from Jim Blandy. We've
> come up with the following approach:
>
> - A complete, typed Rust implementation of the CDP protocol messages and
> (de)serialization lives in the "cdp" crate [2], automatically generated
> from the protocol's JSON specification [3] using a build script (this
> happens transparently as part of the normal Cargo compilation process).
> This comes with Rustdoc API documentation of all messages/types in the
> protocol [4] including textual descriptions bundled with the specification
> JSON. The cdp crate will likely track the Chrome stable release for which
> version of the protocol is supported. A maintainers' script exists which
> can find and fetch the appropriate JSON [5].
>
> - The "tokio-cdp" crate [6] builds on the types and (de)serialization
> implementation in the cdp crate to provide a server implementation built on
> the Tokio asynchronous I/O system. The server side provides traits for
> consuming incoming CDP RPC commands, executing them concurrently and
> sending back responses, and simultaneously pushing events to the client.
> They are generic over the underlying transport, so the same backend
> implementation could provide support for "remote" clients plugging in over
> HTTP/WebSockets/JSON or, for example, a browser-local client communicating
> over IPDL.
>
> - In Servo, a new component plugs into the cdp and tokio-cdp crates and
> acts on behalf of connected CDP clients in response to their commands,
> communicating with the rest of the Servo constellation. This server is
> disabled by default and can be started by passing a "--cdp" flag to the
> Servo binary, binding a TCP listener to the loopback interface at the
> standard CDP port 9222 (a different port can be specified as an option to
> the flag).
>
> - The implementation we envision in Firefox/Gecko would act similarly: a
> new Rust component, disabled by default and switched on via a command line
> flag, which binds to a local port and mediates between Gecko internals and
> clients connected via tokio-cdp.
>
> We chose to build this on Rust and the Tokio event loop, along with the
> hyper HTTP library and rust-websocket which plug into Tokio.
>
> Rust and Cargo provide excellent facilities for compile-time code
> generation which integrate transparently into the normal build process,
> avoiding the need to invoke scripts by hand to keep generated artifacts in
> sync. The Rust ecosystem provides libraries such as quote [7] and serde [8]
> which allow us to auto-generate an efficient, typed, and self-contained
> interface for the entire protocol. This moves the complexity of ingesting,
> validating, and extracting information from client messages out of the
> Servo- and Gecko-specific backend implementations, helps to ensure they
> conform correctly to the protocol specification, and provides a structured
> way of upgrading to new protocol versions.
>
> As for Tokio, the event loop and Futures-based model of concurrency it
> offers maps well to the Chrome DevTools Protocol. RPC commands typically
> execute simultaneously, returning responses in order of completion, while
> the server continuously generates events to which the client has
> subscribed. Under Tokio we can spawn multiple lightweight Tasks, dispatch
> messages to them, and multiplex their responses back over the single client
> connection. The Tokio event loop is nicely self-contained to the one or,
> optionally, more threads it is allocated, so the rest of the application
> doesn't need to be aware of it.
>
> Use of Tokio is becoming a standard in the Rust ecosystem---it's worth
> mentioning that Mozilla funds Tokio development [9] and employs some of its
> primary developers. Servo currently depends on an older version of the
> hyper HTTP client/server library, and consequently this is already present
> in the Firefox tree. The current release of hyper is built on top of Tokio,
> so upgrading hyper, either as maintenance or to take advantage of the
> forthcoming HTTP/2 support, would require pulling in Tokio anyway. The
> current release of rust-websocket, from which Servo derives its WebSockets
> implementation, also supports Tokio.
>
> The alternative to Tokio for the networking layer would likely be to build
> on top of Necko. This presents its own share of problems.
>
> No Rust bindings to Necko currently exist, so writing those in the first
> place would become a prerequisite for the rest of the work, and may require
> integration with technologies like XPCOM which also lacks Rust support.
> Then rewriting the CDP networking layer on top of Necko would duplicate
> effort between the CDP implementations for Gecko and Servo, the latter of
> which does not presently use Necko and may not be able to take it on as a
> dependency.
>
> Furthermore, it is my understanding through conversations with others that
> while Necko's HTTP client implementation is mature, what HTTP *server*
> implementation exists is bare-bones, and that at present there is no
> support for upgrading HTTP connections to WebSockets in order to implement
> a WebSocket server. Then building on top of Necko would entail effectively
> developing a new library within Necko, when we could instead adopt existing
> technologies (Tokio, hyper, rust-websocket) which are already in use by
> other Mozilla projects and the surrounding Rust ecosystem.
I assume this is going to involve TLS (generally this is a requirement for
H2). In Firefox, this is done with NSS. Does Tokio/Hyper cleanly separate
out the TLS stack so that you can do that?
-Ekr
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
