I presume that when a test is disabled a bug is filed and triaged within
the responsible team as any regular bug. Only that way we don't forget
and push on fixing it and returning back to the wheel.
Are there also some data or stats how often tests having a strong orange
factor catch actual regressions? I.e. fail a different way than the
filed "intermittent" one and uncover a real bug leading to a patch back
out or filing of a regular functionality regression bug. If that number
is found to be high(ish) for a test, the priority of fixing it after its
disabling should be raised.
-hb-
On 07/03/2017 17:10, jma...@mozilla.com wrote:
On Tuesday, March 7, 2017 at 10:37:12 AM UTC-5, Boris Zbarsky wrote:
On 3/7/17 9:26 AM, jma...@mozilla.com wrote:
We find that we are fixing 35% of the bugs and disabling 23% of them.
Is there a credible plan for reenabling the ones we disable?
-Boris
Great question, we do not have a credible plan, but we will have a quick way to
see what is disabled:
https://goo.gl/EfDKxY
I would like to build a dashboard when possible to outline per component which
test cases exist, which are disabled, and what related intermittent bugs there
are. Part of annotating moz.build files with BUG_COMPONENTS is to make it
easier to associate all test cases with components whoch would help with a
dashboard.
Do you have suggestions for how to ensure we keep up with the disabled tests?
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
