This looks really great! How many alerts is this thing going to generate? Could you have a mailing list that just gets all of the telemetry alerts email and then people could subscribe to it if they wanted? (Technically, I suppose I could make a mailing list myself and land a patch to subscribe the list to every one of these...) I'm not sure I really want to have my email address distributed with every copy of the Firefox source code. :)
Andrew ----- Original Message ----- > We, as in the performance team, are currently working on an automatic > alerting system for Telemetry's data. At the moment we can detect sensible > changes in the distributions of the collected histograms and we would like > to notify the authors of the histograms with an e-mail. > > The e-mail will include the name of the histogram, the date of the regression > and a plot that shows two histograms: the reference and the regression > distribution. The regression histogram shows the distribution of the > histogram taken the day it changed, while the reference histogram shows how > the distribution should look like based on the previous days. In the future > the e-mail will include a link to Telemetry's dashboard which will likely > show some more information. > > Since using 'hg blame' or a similar approach to pinpoint the author of the > histogram doesn't always yield the right match (see Bug 1037494), we are > likely going to ask the interested authors to manually add their e-mail > address to toolkit/components/telemetry/Histograms.json. > > The question to the histogram authors now is if receiving alerts through > e-mails is OK or they would prefer something else. Note that a great deal of > care has been taken to ensure that there the number of false positives is > very close to 0, i.e. if you receive an alert then the distribution has > clearly changed and the effect size is big enough that it can be considered > more than significant. So don't expect your mailbox to be flooded with > useless alerts. > > Roberto > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
