On Thu, Jun 20, 2024 at 8:50 AM Piotr P. Karwasz
<piotr.karw...@gmail.com> wrote:
>
> Hi Gary,
>
> On Thu, 20 Jun 2024 at 13:55, Gary Gregory <garydgreg...@gmail.com> wrote:
> > I'll give you all my -1 POV here:
>
> I share some of your concerns.
> My main motivation for this experiment with Develocity is not to save
> up CI time, but to improve its responsiveness.
>
> From my contributions to other projects I learned that each project
> has some ("arbitrary") set of PR checks (formatting, unit tests,
> integration tests).
> Some of these checks fail fast enough so that I am able to fix them in
> the time I have available for that project. If those checks fail
> later, I'll fix them a couple of days (or weeks) later.
>
> We could configure PTS as a fail-fast mechanism:
>
> * for each PR we first run with PTS to see if it can find failing tests,
> * if that check succeeds we run the whole suite.
>
> If the PTS algorithm is sound, we might provide feedback to users in
> less than 10 minutes. Worst case scenario, they'll learn that the
> build failed later.
>
> What do you think?Thank you for your thoughtful comments Piotr :) This is, IMO, both a false economy and frustration in making for contributors. Let me explain: This will cause more iterations between contributors and the CI/us to approve runs and wait for CI runs. By default IIRC, one of us has to _approve_ a PR before it is run, which I believe is the correct behavior. A contributor _should_ run a full local build _before_ pushing a PR IMO and not rely on the CI as their build machine. The workflow I encourage (for example in Apache Commons) is: - We make sure a Maven POM has a default goal containing all build checks and we document to ask users to run 'mvn' (by itself) _before_ pushing a PR. Having a default goal is simple. avoids missing any checks, and allows us to edit the behavior without changing build and PR instructions. - This process allows the contributors to fix everything locally _before_ pushing and waiting for the initial and subsequent PR run approval. This saves time and encourages contributors to clean up or complete their PR before bothering to push. There is nothing we can do to avoid getting broken PRs of course. Another missing piece is that a PR should fail if it decreases code coverage or has missing tests. One way I found to enforce this in Commons is to use JaCoCo's metrics checks; it's not perfect but it sets a bar a PR must pass. Gary > > Piotr
