From the earlier thread about aurora uplifts on firefox-dev (I'm
crossposting to m.d.platform, please follow-up on fx-dev because that's
where the original thread was.):
On 22/11/2015 15:41, L. David Baron wrote:
Is a significant part of the instability on aurora related to things
that land on aurora, rather than things that are unstable on nightly
and then are uplifted to aurora every 6 weeks?
I think if we're worried about crashes on aurora, we should be
putting more resources into quickly filing and quickly fixing (or
backing out) crash regressions on both nightly and aurora. My
understanding is that we spend most of our topcrash-chasing
resources on stages later in the release train.
[...]
Letting low quality hang around on nightly and aurora for extended
periods of time just improves the chances that some of it will get
through to release. If we want to address issues of quality on
those channels, I'd rather use a project management process that
looks more like sheriffing than one that looks like approvals.
On 23/11/2015 14:33, Robert Kaiser wrote:
I fully agree, but we don't have the resources. My job is release
quality (including stability), so I concentrate first on what is on or
near release, and only as time allows take a look at Dev Edition and
Nightly - which due to frequent fires on beta is very rare.
In addition, we recently lost the only developer that was focused on
looking at crashes (dmajor) with no replacement in sight. He looked at
and found a number of issues on Nightly pretty fast, esp. as he knew
what things mean when looking at code. I'm convinced we'll need
someone like that again - plus additional people to look at data and
identify issues.
What's the best way forward from this? It sounds to me like there are a
number of things we could do (not necessarily exclusive):
- "a project management process that looks more like sheriffing than one
that looks like approvals" -- dbaron, can you expand on that? What do
you have in mind?
- increase our focus on automated testing. I've heard suggestions that
we should hire people to just write more tests, or force more landings
to come with tests.
- increase our use of static analysis / linting / automated scanning of
things. I posted something else for the JS side of this. On the compiled
code side of things, Sylvestre, do you know if any of the tools we're
currently using are being run on checkins / every day to detect *newly
introduced* problems specifically, and to file bugs on them (which we
should be able to automate with the bugzilla API, I think)? Is there
anything we can ramp up / improve there?
- increase the manual testing we do of things that happen on Nightly.
Ryan, can you comment on this? It seems we have the possibility to get
extra QE attention for "new" projects we land on Nightly. Are there
(other) people focused on quality/smoketesting things on nightly?
- I notice from bugmail that we have days when new volunteers help
verify fixed bugs, but this mostly happens when those bugs are on the
release/beta channel at that point. How stretched are we for capacity to
do that on Nightly instead? Mike, how easy would it be to adapt the tool
we have for triage to give people bugs to verify? Same question for
things like filing topcrashers and getting regression ranges / STR for
them... Robert, are there simple instructions for how to examine and
file topcrashes on MDN/wikimo? If not, would you be able to write some?
~ Gijs
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
