I've created a meta bug to track efforts to improve the UX for this use case: https://bugzilla.mozilla.org/show_bug.cgi?id=1286693
Please feel free to add suggestions there or in dependent bugs. On Mon, Jul 11, 2016 at 11:30 AM, allassopraise . <allassopra...@gmail.com> wrote: > The suggestions for improving the way treeherder reports are good. > > Nevertheless, for now, I still would like some documentation or > pointers on how to interpret what is there, but most especially how to > know whether an error is due to the changeset that's been pushed. > > *Somebody* is obviously able to do it, as I can tell when a patch I've > written gets pushed, then backed out due to results on treeherder. > > On 7/11/16, Jonathan Griffin <jgrif...@mozilla.com> wrote: > > Yes, a clear Yes/No answer for Try pushes is something we'd like to be > able > > to report. > > > > Currently, interpreting Try results can be hard for a few reasons: > > > > 1 - intermittent test failures make it hard to separate failures your > push > > has caused vs pre-existing failures > > 2 - people sometimes push to try from bad heads (like a mozilla-inbound > > push that will eventually get backed out) > > 3 - it's not always clear whether you should care about try failures of > > Tier 2 jobs > > > > For 1, as you noted, we have started implementing automatic > classification > > in Treeherder. We're expanding this to cover more types of failures; see > > bug 1241940. The goal is to have it able to identify upwards of 80% of > > known failures automatically, which should make interpreting try results > > considerably easier. > > > > For 2, making sure you push to try only from the current tip of m-c will > > avoid this problem, or using MozReview's autoland to try, which will > > automatically do the same thing. > > > > For 3, I think we should distinguish tiers better in Treeherder, and > > include some info about tolerance for failures in the UI. > > > > Jonathan > > > > On Mon, Jul 11, 2016 at 4:18 AM, Nicolas B. Pierron < > > nicolas.b.pier...@mozilla.com> wrote: > > > >> On 07/10/2016 06:48 PM, allassopraise . wrote: > >> > >>> For example, I believe the orange and red links indicate an error, but > >>> how do I know if they pertain to my push or not? > >>> > >> > >> I have the same issue, and in many cases I feel clueless about the > >> oranges > >> & reds. At the end I end-up pushing to inbound, even if I feel that I > >> had > >> a higher number of oranges / reds on try than I would expect to see on > >> inbound. > >> > >> I see that we already have a classifier which automatically mark some of > >> the intermittent, and if this tool were able to classify the remaining > >> issue, this would be quite helpful. > >> > >> I guess, what I am missing with Try pushes, is a clear Yes/No answer. > >> > >> -- > >> Nicolas B. Pierron > >> > >> > >> _______________________________________________ > >> dev-builds mailing list > >> dev-builds@lists.mozilla.org > >> https://lists.mozilla.org/listinfo/dev-builds > >> > > >
_______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
