My preference would be to have ALL test targets be mandatory to merge any PR, but in my opinion that would require some sort of plan to fix all flickering tests. Currently, we have a person assigned to monitor CI everyday who then files Jira tickets against any test that intermittently fails, but presently the vast majority of those Jira tickets remain unassigned and are never fixed.
A couple of options right now. Make both stress-new-tests required for merge: 1) but only for new tests 2) with zero tolerance for flickering tests -- which means all PRs not directly associated with fixing flickering tests are frozen indefinitely until ALL Jira tickets for flickering tests are fixed, and this process is then repeated every time we have an intermittent failure in CI -Kirk On Thu, Jun 10, 2021 at 10:56 AM Dan Smith <dasm...@vmware.com> wrote: > I'm also -0 on this one. I personally kinda like having the merge button > disabled on my PRs until all the checks are passing so I don't have to > think as much, and I'm not sure this problem is going to come up again > soon. But I'm willing go with disabling it. > > -Dan > ________________________________ > From: Mark Hanson <hans...@vmware.com> > Sent: Thursday, June 10, 2021 10:15 AM > To: dev@geode.apache.org <dev@geode.apache.org> > Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from > PRs > > +1 to new report from Owen > +1 to re-introduce the stress-test > > Great ideas Naba! > > On 6/10/21, 9:50 AM, "Nabarun Nag" <n...@vmware.com> wrote: > > Hi all, > > > * We need to discuss how to prevent more flaky tests to be > introduced now that stress-test is not mandatory for PRs to be merged? > Reviewers checking the PR must check the tests failing in stress test and > if it is a test that has been introduced or changed in the PR, the PR must > be blocked with a change request or rejected. > * Also, in my opinion, we need to re-introduce the stress test as > a mandatory check for PRs to be merged once the flaky test percentage has > been reduced. > > Owen, will it be possible to put out a list of all the flaky tests and > we can try to get these resolved collectively as a community. (results of > the mass tests maybe). Thank you. > > > Regards > Nabarun Nag > ________________________________ > From: Kirk Lund <kl...@apache.org> > Sent: Thursday, June 10, 2021 9:37 AM > To: dev@geode.apache.org <dev@geode.apache.org> > Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement > from PRs > > Ok, I wanted to give this discussion another night and we still have > consensus for making both stress-new-test non-required. > > I just filed PR #6602 < > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fpull%2F6602&data=04%7C01%7Cdasmith%40vmware.com%7C58dd1b5852174343eebb08d92c335953%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637589421336670457%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2Fuc8WXORNjjYjFKNgGjqMdS%2FhK9FGFBepNCNqmfp1GQ%3D&reserved=0> > to change > stress-new-test from required to non-required. Please review! > > On Wed, Jun 9, 2021 at 2:11 PM Anthony Baker <bak...@vmware.com> > wrote: > > > If we have consensus, no need to VOTE. > > > > > On Jun 9, 2021, at 12:52 PM, Owen Nichols <onich...@vmware.com> > wrote: > > > > > > Ok, I'm on board with changing stress-new-test from a required PR > check > > to non-required. It's simple, codeowners still have the final say, > and it > > neatly avoids the whole topic of overrides. Time to take a [VOTE]? > > > > > > > > >
