On Wednesday 2017-03-08 17:15 -0500, Kartikaya Gupta wrote: > On Wed, Mar 8, 2017 at 4:01 PM, <chris.ryan.pea...@gmail.com> wrote: > > In the past I have not always been made aware when my tests were disabled, > > which has lead to me feeling jaded. > > We have a process (in theory) that ensures the relevant people get > notified of tests. The process involves these steps: > 1) There is a moz.build file somewhere in the tree which covers the > tests in question and specifies a BUG_COMPONENT for it > 2) If a test starts failing intermittently, a bug is filed in the > aforementioned component > 3) The component is monitored/triaged regularly by the module owner or team > 4) Whoever triages the bug notifies the individual owner if they're > not already on the bug > > So if for some reason anybody feels like they were not notified when > their tests are being disabled, they should find the link the above > chain where things broke down (e.g. no BUG_COMPONENT for a test, bug > filed in a component other than BUG_COMPONENT, nobody triaging new > bugs, etc.) and do something about it.
As of 5 days ago, "Treeherder Bug Filer" was not using BUG_COMPONENT information. I say this based on: https://bugzilla.mozilla.org/show_bug.cgi?id=1344304 being filed in Core :: Layout despite: > $ ./mach file-info bugzilla-component layout/style/test/test_compute_data_with_start_struct.html > Core :: CSS Parsing and Computation > layout/style/test/test_compute_data_with_start_struct.html (I wish it did use BUG_COMPONENT! That's the main reason I bothered to write good BUG_COMPONENT data for most of layout/*.) > >> and it would be reasonable to expect a fix. > > > > I think it's unreasonable to assume that developers can drop whatever > > they're doing and turn around a fix in a two weeks, given how long these > > things often take to fix, and given that developers often have a > > pre-existing list of other high priority stuff to work on. > > > > In my experience it's not so much that a fix is needed in two weeks, > it's that you need to put in a good-faith effort to respond and start > investigation. Oftentimes it legitimately takes longer than two weeks > to fix intermittents, but I've never had a scenario where I asked for > more time and was denied that. I think it's often reasonable to expect a *backout* of the cause within less than two weeks, although perhaps not if the immediate trigger was changes in test chunking. -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla https://www.mozilla.org/ 𝄂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914)
signature.asc
Description: PGP signature
_______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
