>
> Do we have a full clear vision of what is left until Beta as problems
> with the flaky tests? What is the common understanding and approach?
>
I vote we start pushing anything that doesn't a) block user testing and b)
risk an API change out from blocking beta to being part of the beta phase
so
> > Good point Jordan re: flaky test being either implying API instability or
> > blocker to ability to beta test.
+1
> While I agree that we should use the Apache infrastructure as > the canonical
> infrastructure, failures in both (or any) environment matter > when it comes
> to flaky tests.
I don't know how hard to do it would be, but I think it would be great to
have a CI job to repeatedly run a specific test, so we can check if it's
flaky. We could systematically run it for new tests, or for existing tests
related to what we are modifying.
Using this CI job to run a suspicious test
> - No flaky tests according to Jenkins or CircleCI? Also, some people run
> the free tier, others take advantage of premium CircleCI. What should be
> the framework?
It would be good to have a common understanding of this; my current mental
model is
1) Jenkins
2) Circle CI Free tear unit tests
Good point Jordan re: flaky test being either implying API instability or
blocker to ability to beta test.
On Thu, May 28, 2020 at 12:56 PM Jordan West wrote:
> > On Wed, May 27, 2020 at 5:13 PM Ekaterina Dimitrova <
> > ekaterina.dimitr...@datastax.com> wrote:
>
> > - No flaky tests according
> On Wed, May 27, 2020 at 5:13 PM Ekaterina Dimitrova <
> ekaterina.dimitr...@datastax.com> wrote:
> - No flaky tests according to Jenkins or CircleCI? Also, some people run
> > the free tier, others take advantage of premium CircleCI. What should be
> > the framework?
While I agree that we shou
Agree re: 15299.
This thread is about pushing out flaky tests and what we define as that
cohort as I understand it.
On Thu, May 28, 2020 at 7:59 AM Ekaterina Dimitrova
wrote:
> CASSANDRA-15299 - All interface-related still open tickets are blockers.
> My point was that they are already just a
CASSANDRA-15299 - All interface-related still open tickets are blockers.
My point was that they are already just a few, looking into Jira. So except
them, flaky tests are really a thing that requires attention.
Also, I agree with Mick that it’s good to have a plan and opened Jira
tickets earlier
> I have the feeling that the thing that prevents us primarily from
cutting beta at the moment is flaky tests
CASSANDRA-15299 is still in progress and I think we have to consider it a
blocker, given that beta "should be interface-stable, so that consumers do not
have to incur any code changes on
>
> Most of these were fixed in CASSANDRA-15622
> But the remaining failures are from the use of
> `FBUtilities.getLocalAddress()` and `InetAddress.getLocalHost()`. It
> affects ci-cassandra because the agents need their public ip so the
> master can reach them.
>
> Some help with how best to fix t
> > So my idea was to suggest to start tracking an exact Jenkins report maybe?
>
> Basing our point of view on the canonical test runs on apache infra makes
> sense to me, assuming that infra is behaving these days. :) Pretty sure
> Mick got that in working order.
It's definitely closing in. Runn
>
> So my idea was to suggest to start tracking an exact Jenkins report maybe?
Basing our point of view on the canonical test runs on apache infra makes
sense to me, assuming that infra is behaving these days. :) Pretty sure
Mick got that in working order.
At least for me, what I learned in the p
Dear all,
I spent some time these days looking into the Release Lifecycle document.
As we keep on saying we approach Beta based on the Jira board, I was
curious what is the exact borderline to cut it.
Looking at all the latest reports (thanks to everyone who was working on
that; I think having an
13 matches
Mail list logo
