2017-06-02 18:40 GMT-04:00 Ehsan Akhgari :
> On 06/02/2017 06:27 PM, Dustin Mitchell wrote:
>>
>> It could go two ways:
>>
>> 1. "Just try it" again and see what breaks. This runs extra jobs, but
>> has the benefit of finding any accidental bustage added in the fix; or
>
> At the cost of forcing yo
On 06/02/2017 06:27 PM, Dustin Mitchell wrote:
It could go two ways:
1. "Just try it" again and see what breaks. This runs extra jobs, but
has the benefit of finding any accidental bustage added in the fix; or
At the cost of forcing you filter through which oranges are your fault
and which ones
It could go two ways:
1. "Just try it" again and see what breaks. This runs extra jobs, but
has the benefit of finding any accidental bustage added in the fix; or
2. "try this" listing the M jobs that failed (or some simpler-to-write
superset like "all chunks of OS X debug mochitest-chrome").
It'
I haven't really read the thread in too much detail so I apologize if
what I'll say below is silly or covered elsewhere, but since there seems
to be a proposal around replacing the try syntax with machines figuring
out what jobs are needed for a push (presumably based on what changed in
the pus
I wrote up
https://docs.google.com/document/d/1kEEUC3ETCIerOxYDoRRZSEFspunJgU331CX2wwkFZ_8/edit
It's not a particularly ambitious text, aiming to capture a consensus
rather than propose something new. It's also worth repeating that this
is not a proposal for a big new project, but a design strat
2017-06-01 10:54 GMT-04:00 Andrew Halberstadt :
> Speaking for myself I agree that it seems unlikely we'll be able to both
> reduce machine consumption and increase test coverage, and I think that's
> fine. When I read dustin's proposal, I see:
>
> #1. The interface to try is terrible, we should fi
Speaking for myself I agree that it seems unlikely we'll be able to both
reduce machine consumption and increase test coverage, and I think that's
fine. When I read dustin's proposal, I see:
#1. The interface to try is terrible, we should fix that
#2. There is some scheduling we could be doing aut
On Wed, May 31, 2017 at 11:08 AM, Dustin Mitchell
wrote:
>
>
> So the proposal addresses three issues:
> - frustrating and ineffective try user interface
> - high load and consequent long wait times
> - elevated backout rate
>
Let me try and repeat back the assumptions/assertions I've heard,
On Wed, May 31, 2017 at 1:31 PM, Gregory Szorc wrote:
>
>> There is a facility in the tree for tagging tests (at least some suites)
>> and running only tests with certain tags. However, that doesn't help on try
>> very much because you can't tell try to run only a certain set of tags.
>>
>
> Not
On Wed, May 31, 2017 at 6:26 AM, Benjamin Smedberg
wrote:
> I don't know if I'm the typical use-case, but the big problem for me is
> that when I change something such as plugins, the jobs as currently
> bucketed don't help much. There are reftests, crashtests, mochitest-plain,
> and mochitest-br
2017-05-31 9:26 GMT-04:00 Benjamin Smedberg :
> Your ultimate goal is to save $$, and my ultimate goal is to reduce the time
> to results to make the developer cycle faster (let's measure round-trip
> times in minutes instead of hours). I don't believe that running subsets of
> the current jobs wil
I don't know if I'm the typical use-case, but the big problem for me is
that when I change something such as plugins, the jobs as currently
bucketed don't help much. There are reftests, crashtests, mochitest-plain,
and mochitest-browser-chrome which test plugin code paths, and the
splitting means t
I think this topic is big enough already without broadening it into
"how can we make automation better". But getting some data from the
survey sounds great! Maybe it makes sense to get down to the core
question we have here:
When you push to try, how often do you want:
* to run every job relevan
On Tue, May 30, 2017 at 05:25:20PM -0700, Gregory Szorc wrote:
> On Thu, May 11, 2017 at 10:05 AM, wrote:
>
> > Background:
> > https://bugzilla.mozilla.org/show_bug.cgi?id=1359942
> >
> > As jobs move to taskcluster, we have an improved opportunity to do some
> > smarter scheduling of what jobs
On Thu, May 11, 2017 at 10:05 AM, wrote:
> Background:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1359942
>
> As jobs move to taskcluster, we have an improved opportunity to do some
> smarter scheduling of what jobs to run on what sort of push. Of course,
> it's a thorny subject: optimizing
On Fri, May 26, 2017 at 06:35:42PM -0400, Dustin Mitchell wrote:
> 2017-05-26 18:29 GMT-04:00 Mike Hommey :
> > Please make it so that it's possible to figure out why some job has run
> > or not. Currently, it's near impossible.
>
> Yep. According to my list, that's reason #34 to call the legacy t
2017-05-26 18:29 GMT-04:00 Mike Hommey :
> Please make it so that it's possible to figure out why some job has run
> or not. Currently, it's near impossible.
Yep. According to my list, that's reason #34 to call the legacy try
parser[*] "trying my patience".
Dustin
[*] reason #13 is that there ar
On Fri, May 26, 2017 at 06:20:33PM -0400, Dustin Mitchell wrote:
> Andrew, I apologize for not giving credit -- the "try this" variant is
> intended to be exactly what you've proposed in that blog entry.
>
> I like the idea of doing the processing client-side. I think we will
> still want awarene
Andrew, I apologize for not giving credit -- the "try this" variant is
intended to be exactly what you've proposed in that blog entry.
I like the idea of doing the processing client-side. I think we will
still want awareness of the three options in taskcluster/, though:
- "there is no try" is ba
I'm very interested in the try aspect of this (please CC me on relevant
bugs).
I believe there is a relatively simple way taskcluster can wash its hand of
try syntax completely and instead just use a list of tasks (or other
similar data structure). We could still provide a legacy try syntax to
peo
Heres my thoughts:
* High level:
** I like the idea of letting cringe-worthy `try: ` obsolete out as a
bad idea going forward
** I like the idea of annotations of affects in moz.build files.
I want to avoid "AFFECTS" doing things like having 30 cases of
"windows" here. (e.g. win10, win64, windows
Greg sent some previous threads on this topic along, and I've been in
some other related conversations. The topic of optimization overlaps
substantially with try, so I'll mix that in too.
There seems to be a reluctance to talk about this face-to-face, but
there's some urgency in that we're spendi
Thanks Ben --
I agree this is probably not the time to go all-in on this, if it's a
major project. But I think it's a good time to come to some consensus
on direction, so that other changes we make along the way support, or
at least do not preclude, solving it later. In particular, I think
we're
Dustin, I am very interested in following up on this. I believe that our
current strategy of running every almost every test on every checkin is
unsustainable long-term, and we're going to have to move to a model where
expensive tests typically are run less frequently. In order to make this
success
Background:
https://bugzilla.mozilla.org/show_bug.cgi?id=1359942
As jobs move to taskcluster, we have an improved opportunity to do some smarter
scheduling of what jobs to run on what sort of push. Of course, it's a thorny
subject: optimizing away a task that should run may let a bad push show
25 matches
Mail list logo
