On 2016-02-27 10:09 AM, Jim Mathies wrote:
On Friday, February 26, 2016 at 5:58:54 PM UTC-6, William Lachance wrote:
Hey,
I wrote up a dashboard for tracking the performance delta between
non-e10s and e10s on the Talos tests on nightly:
https://treeherder.allizom.org/perf.html#/e10s
This is great, thanks!
A couple questions - what are the run values here? For example, "100 base / 139
new", I'm curious what this means.
It refers to the number of times the talos test was run for non-e10s and
e10s, over the entire sample of pushes. 100 base / 139 new == 100 test
runs for non-e10s, 139 test runs for e10s.
Also, if we land a fix or someone lands a patch that causes a regression on
inbound, how long would it take for that to affect values in this dashboard?
We sample talos data for pushes in the last 48 hours, so it would take
that long for the dashboard to be fully updated after the patch is
landed. Some talos tests are rather noisy, so I felt we needed this much
data to be confident in the results on the graph in all cases.
You can always monitor the graphs themselves if you want to get more
up-to-date information (hovering over a row will give you link), and of
course :jmaher and I are monitoring perfherder for large changes here:
https://treeherder.allizom.org/perf.html#/alerts
Will
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
