I've been thinking about how we might improve access to telemetry data. I don't want to get too much into the issues with the current front-end, so suffice it to say that it isn't meeting my group's needs.
I solicited ideas from Justin Lebar and Patrick McManus, and with those I came up with an overview of how those of us in platform engineering would ideally like to be able to access telemetry data. I'd love to hear thoughts and feedback. === At the highest level, we want to make decisions, mostly regarding optimizations, based on feedback from "the wild." We want to know when a change makes something better or worse. We also want to be able to spot regressions. The top priority for exposing telemetry data is an easy-to-use API. Secondarily, there should be a default front-end. An API will allow developers to "innovate at the edges" by building tools to fit their needs. This will also remove the one-size-fits-all requirement from the default front-end. The API should be easier to use than mucking with hadoop/hbase directly. It might be a RESTful JSON API. It should not require people to apply for special privileges. The default front-end should be fast, stable, and flexible. It should be based as much as possible on existing products and frameworks. Being able to modify the display of data as needs change should be easy to do, to avoid long wait times for new views of the data. It should provide generally useful views of the data, breaking down results by build (builds contain dates in their IDs), so we can see how results change from one build to another. We want to be able to see statistical analyses such as standard deviations, and cumulative distribution functions. We would also like to be able to run A/B experiments. This means coming up with a better instrumentation framework for code in the builds, but it also means having a dashboard that understands the results, and can show comparisons between A and B users that otherwise have the same builds. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
