On 2016-02-29 1:52 AM, Chris Peterson wrote:
Will, this dashboard looks great! The current e10s release criteria
allows regressions up to 5% on the following Talos tests. Is it possible
to configure your e10s dashboard to display acceptable (<= 5%)
regressions on these particular tests using another color besides red?
Perhaps yellow to show that the e10s results are worse than non-e10s,
but not failure red?
...
After talking with cpeterson on irc, I came up with a mode for the
dashboard which hides everything except for "regressions blocking the
release" according to the criteria here:
https://wiki.mozilla.org/index.php?title=Electrolysis/Release_Criteria
You can see this view here:
https://treeherder.allizom.org/perf.html#/e10s?showOnlyBlockers=1
Also, mconley suggested being able to compare the results of individual
subtests. You can access this view for any given talos test by hovering
over the line in the comparison and selecting "subtests". This sometimes
give interesting data, for instance on "tps" some pages are clearly
causing more problems than others:
https://treeherder.allizom.org/perf.html#/e10s_comparesubtest?baseSignature=fe016968d213834efd424ca88680cfa7490b6c09&e10sSignature=5c199ff7bd97284c5f3820ba908f92275620cd8b
(notice how aljaazera.net has a consistent ~450% regression!)
Will
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
