Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

From: Mark Hanson > Sent: Thursday, June 10, 2021 10:15 AM > To: dev@geode.apache.org > Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from > PRs > > +1 to new report from Owen > +1 to re-introduce the stress-test > > Great ideas Naba! > > On 6/10/

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

disabling it. -Dan From: Mark Hanson Sent: Thursday, June 10, 2021 10:15 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs +1 to new report from Owen +1 to re-introduce the stress-test Great ideas Naba! On 6/10/21, 9:50 AM, "Nab

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

abarun Nag From: Kirk Lund Sent: Thursday, June 10, 2021 9:37 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs Ok, I wanted to give this discussion another night and we still have consensus for making both stress-ne

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

ards Nabarun Nag From: Kirk Lund Sent: Thursday, June 10, 2021 9:37 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs Ok, I wanted to give this discussion another night and we still have

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

these resolved collectively as a community. (results of the mass tests maybe). Thank you. Regards Nabarun Nag From: Kirk Lund Sent: Thursday, June 10, 2021 9:37 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Ok, I wanted to give this discussion another night and we still have consensus for making both stress-new-test non-required. I just filed PR #6602 to change stress-new-test from required to non-required. Please review! On Wed, Jun 9, 2021 at 2:11 PM Ant

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Count me as -0. I have some concerns, but I’m okay trying this and seeing how it goes. Dale From: Owen Nichols Date: Wednesday, June 9, 2021 at 2:25 PM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs Summarizing this thread so far: In

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Summarizing this thread so far: In favor of making stress-new non-required: Kirk Mark Myself In favor of making all PR checks non-required: Jake In favor of hashing out a more nuanced balance between making it possible (but not too easy) to ignore stress-new failures: Donal Dale Maybe that's r

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

If we have consensus, no need to VOTE. > On Jun 9, 2021, at 12:52 PM, Owen Nichols wrote: > > Ok, I'm on board with changing stress-new-test from a required PR check to > non-required. It's simple, codeowners still have the final say, and it > neatly avoids the whole topic of overrides. Time

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

results we don’t trust. Dale From: Kirk Lund Date: Wednesday, June 9, 2021 at 9:59 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requ

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

In this scenario, the extra burden on code owners arises only at the committer’s request. Dale From: Owen Nichols Date: Wednesday, June 9, 2021 at 11:15 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requi

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

9, 2021 at 9:59 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs I do like the suggestions offered up by Dale and would encourage (or even plead) with my fellow contributors to consider the

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

From: Owen Nichols Date: Wednesday, June 9, 2021 at 11:15 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs This would substantially increase the burden on codeowners, because now in addition to looking at the code itself, they

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Date: Wednesday, June 9, 2021 at 11:15 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs This would substantially increase the burden on codeowners, because now in addition to looking at the code itself, they would have to wait for any

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

und Date: Wednesday, June 9, 2021 at 9:59 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs I do like the suggestions offered up by Dale and would encourage (or even

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

frame as: We have too many tests whose results we don’t trust. Dale From: Kirk Lund Date: Wednesday, June 9, 2021 at 9:59 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

te: Wednesday, June 9, 2021 at 9:59 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs I do like the suggestions offered up by Dale and would encourage (or even plead) with my fellow contributors to consider th

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

ne of this fixes the underlying problem, which I’d frame as: We have too many tests whose results we don’t trust. Dale From: Kirk Lund Date: Wednesday, June 9, 2021 at 9:59 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from P

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

ce. From: Mark Hanson Sent: Wednesday, June 9, 2021 10:16 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs One other thing to think about is perhaps having a rotating team to deal with flaky tests, a small team co

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

One other thing to think about is perhaps having a rotating team to deal with flaky tests, a small team commissioned every three or 6 months to clear out flaky tests for 1 month. It is good experience Thanks, Mark On 6/9/21, 10:04 AM, "Mark Hanson" wrote: One other thing is that we have

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

: We have too many tests whose results we don’t trust. Dale From: Kirk Lund Date: Wednesday, June 9, 2021 at 9:59 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs I do like the suggestions offered up by Dale and would encourage (or even

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

One other thing is that we have code owners. If the PR submitter decides to ignore the stress new test, the code owner can still request a fix. That is probably a sufficient process. On 6/9/21, 10:02 AM, "Mark Hanson" wrote: I agree, I am willing to concede this discussion, as long as we

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

I agree, I am willing to concede this discussion, as long as we are judicious and specifically only use it when it is not our test that is failing. It is a real problem. Thanks, Mark On 6/9/21, 9:46 AM, "Kirk Lund" wrote: I did think about splitting up dunit tests, but I believe testE

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

I do like the suggestions offered up by Dale and would encourage (or even plead) with my fellow contributors to consider these: * Allow code owners to override the block, if they can be convinced the > override is justified. > * Exclude troublesome tests from stress test runs, either via >

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

I did think about splitting up dunit tests, but I believe testEventIdOutOfOrderInPartitionRegionSingleHop will remain flaky even if I move it to a new dunit test. No matter how you dice it up, we end up with a PR that cannot be merged to develop unless you get lucky after running stress-new-test ma

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

I think the basic problem is that we have too much tech debt in the form of dunit tests. We are proposing all of these "workarounds" to avoid dealing with the core problem. On 6/8/21, 12:09 PM, "Dan Smith" wrote: Would it be possible to just split that test up into multiple classes? It s

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Would it be possible to just split that test up into multiple classes? It sounds like the issue is that there is so many flaky tests in that class that you can't fix them all in one PR, which might indicate it's too big. If we can't get StressNewTest to pass - that means our builds are failing m

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Maybe we can find a way to relax the requirement, or to allow addressing specific situations like the tangle you find yourself in. Removing the requirement altogether feels overly broad. I fear it would allow us to quietly disregard all intermittent test failures, and I think we already quietly

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

I think repeated tests shouldn’t be a blocker to merging for the reasons outlined below. A committer that is a good steward for the project should be allowed to make the judgement call when merging a PR. We have placed too many rigid processes in place that eliminated the good judgement of commi

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

Thanks Kirk for tackling some of our flakiest tests! I agree, we don't want to discourage anyone interested in paying down tech debt. The Geode community has spoken clearly against bypassing or weakening required PR checks, so relaxing requirements in general might be a tough sell, but I'm cur

Re: [DISCUSS] Remove stress-new-test-openjdk11 requirement from PRs

I understand the challenge, but I disagree. It is only through requirement that we keep new flakey tests out. While I don't think one should have to fix all the flaky tests to get their unrelated change in, I think it serves a purpose. IMHO, the problems that you are seeing are indications that