Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Tibor Digana
Hello Maximilian, So now the next step is to break the traditional dependencies in Maven and isolate the services via web-services, e.g. JAX-RS or JAX-WS and you would not "touch" the POMs. You need to use Logstash, Kibana, Elasticsearch, and Zipkin because the logs won't be aggregated without the

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Romain Manni-Bucau
Le sam. 14 sept. 2019 à 22:17, Alexander Ashitkin a écrit : > Let us evaluate this approach. But if we go extension way, it will be not > so big motivation to make it part of maven. and i'm not sure what are long > term strategy for maven, but without incremental builld it becomes less and > less

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Alexander Ashitkin
Let us evaluate this approach. But if we go extension way, it will be not so big motivation to make it part of maven. and i'm not sure what are long term strategy for maven, but without incremental builld it becomes less and less attractive in our multi-branched world Thank you On 2019/09/14 0

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Alexander Ashitkin
HI Robert seems to your thinking matches to our own one. Indeed, the question is - should be executed this goal or not? But in current plugin-api limitations not a plugin-developer but only project developer can understand should this goal run or not because of side effects each plugin might hav

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

2019-09-14 Thread Romain Manni-Bucau
Le sam. 14 sept. 2019 à 19:11, Maximilian Novikov a écrit : > Classification: For internal use only > > It's moving to off-topic. > > Monorepo != single build > Agree but then how your figures relates to your repo? You exposed a single build (or equivalent with downstream jobs) state. Monorepo

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

2019-09-14 Thread Romain Manni-Bucau
Le sam. 14 sept. 2019 à 19:34, Maximilian Novikov a écrit : > Classification: For internal use only > > Ok, looks like this is the only option for us: create extension and > override MojoExecutor. > > > The only challenge is an exhaustive test suite since your current impl > can easily fake a pas

RE: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Maximilian Novikov
Tibor, We understand your position. We moved from separated SCM to one SCM. We can move back, but we don't want this. In single SCM we like: 1. Atomic commits 2. Single point of responsibility. If someone makes incompatible change in shared library, he is responsible to update all usages. At f

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Tibor Digana
Alexander, Enrico is really right. Today it is Microservices and there every microservice is in a separate SCM repo. It was just only an example with Microservices but in my experiences you can always find the lower bound modules in the hierary which do not change so much and segragate them in ano

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Tibor Digana
Robert, I understand the discussion. There is the requirement (1) to cache the targets and second discussion is to (2) switch on/off unchanged modules in multi-module project. I had used the (1) in the company Software AG and it was really unpredictable build with hunders POMs in project. I would

RE: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

2019-09-14 Thread Maximilian Novikov
Classification: For internal use only Ok, looks like this is the only option for us: create extension and override MojoExecutor. > The only challenge is an exhaustive test suite since your current impl can > easily fake a passing build (as gradle does today if you don't disable the > daemon an

RE: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

2019-09-14 Thread Maximilian Novikov
Classification: For internal use only It's moving to off-topic. Monorepo != single build Monorepo != monolithic application We moved from poly-repo to monorepo 3 years ago and see value in this. Let's say that this approach exists. It has own benefits/drawbacks comparing to poly-repo. -Ori

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Romain Manni-Bucau
Hope I didnt miss it but how monorepo=single build? It is working well to not have a common parent too and is unlinked to monorepo which uses local relative paths in general (at least in the references you quoted which are also not about java ;)). Unrelated to making maven better at incremental b

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Robert Scholte
https://issues.apache.org/jira/browse/MPLUGIN-350 is the issue to start with. Please read all the comments, because my original thought won't work. thanks, Robert On Sat, 14 Sep 2019 17:10:13 +0200, Alexander Ashitkin wrote: We checked and price of 550$ per user makes us think twice of w

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Alexander Ashitkin
We checked and price of 550$ per user makes us think twice of what's the best way to proceed here :-) Regarding plugin api - yes, changes are desirable to make maven model cache-friendly. Both in plugin invocation model and Mojo#execute input/output apis. But it is possible to work with current

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Alexander Ashitkin
Sorry, i think Jenkins is irrelevant here at all. we need solution which works everythere - on workstations, from commandline, on other build servers, in intellij, etc. Any solution which is worth to discuss must be Jenkins agnostic honestly. Also, FYI we don't rely on target directory state at

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Alexander Ashitkin
HI Enrico Thanks for feedback. that's a side discussion for best approach for projects layouts. Monorepo has own own advocates and it is easy to find posts describing why google, microsoft or facebook go monorepo. Unlike of way of thought, we are ready to go globally in case of emergency scenar

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Alexander Ashitkin
Hi Laird yes, in this case B will be rebuilt - that's on of the basic requirements. Thanks in advance On 2019/09/14 06:29:00, Laird Nelson wrote: > On Fri, Sep 13, 2019 at 11:01 PM Alexander Ashitkin < > alexander.ashit...@db.com> wrote: > > > This feature is true incremental build – you don’t

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Robert Scholte
Tibor, it seems like you're missing the bigger picture. The question is similar to what we've discussed in the past: can we define if surefire should be executed or not? We should define incremental builds as "should a goal be executed or not?", e.g. based on the results of the previous buil

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Tibor Digana
oh yeah, exactly opposite. Jenkins has several ways to create Maven build configuration and it knows where the repo and workspace is, it knows where to store the archive, it knows when the build failed. We cannot take the responsibility because the build may fail for whatever reason and we do not k

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Romain Manni-Bucau
Tibor, maven is the only one with the logic to give any cache the data it needs. Jenkins alone can't since it does not own the reactor nor plugin I/O values. Le sam. 14 sept. 2019 à 12:45, Tibor Digana a écrit : > But I do not understand why the Maven should be responsible for the project > cahe

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Tibor Digana
But I do not understand why the Maven should be responsible for the project cahe control/management of "/target" directories. It is a responsibility of the build manager which is the Jenkins. The Jenkins has the ability to archive files and such property already exists in the Jenkins. So the Jenki

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Robert Scholte
On Fri, 13 Sep 2019 23:37:15 +0200, Romain Manni-Bucau wrote: There are multiple possible incremental support: 1. Scm related: do a status and rebuild downstream reactor 2. Full and module build graph: seems it is the one you target, ie bypass modules without change. Note that it only works

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Romain Manni-Bucau
Le sam. 14 sept. 2019 à 08:00, Alexander Ashitkin a écrit : > Indeed we have a kind of the option 2 with variations. Current > implementation is opt-in feature driven by configuration with some metadata > of required cache behavior and hints. > > Maven extensions is the option, but we would love

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

2019-09-14 Thread Enrico Olivelli
I feel that in general having an huge monolithic project is kind of a project-smell. Btw I have some big project with 100+ modules so I can see your pain. In the daywork experience a single developer doesn't work on all of the modules but usually you touch 1-2 modules and maybe some integration/sys