[
https://issues.apache.org/jira/browse/MRESOLVER-229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17453998#comment-17453998
]
wei cai commented on MRESOLVER-229:
-----------------------------------
[~michael-o]
* have you evaluated then solution proposed in MRESOLVER-7?
The PR in this ticket has been reverted. I agree this is the more native
solution, however it's kinda solution like: "resolve partially -> download
partially in parallel -> resolver partially... ", it does not make full use of
threads.
* Since you are caching the resolution of dependencies how do you know that
this cache is uptodate or stale?
It does not matter whether the cache is uptodate or stale. My solution is like
"warm-up dependencies in parallel based on the artifacts.json" -> “maven
resolver to resolve in original way”, so event cache is stale, build still
pass as there is 2nd chance that maven will resolve the missing parts.
It's kinda a layer built on top of maven-resolver, with such layer, we can
download most of artifacts in parallel and besides it make full use of threads
at least in the warm-up process.
* currently there is works for a build cache within Maven. Have you read about
it? Does this overlap with your work?
May I know which ticket or which part of code you are referring?
> Improve the download speed by downloading poms & jars in parallel as a warm-up
> ------------------------------------------------------------------------------
>
> Key: MRESOLVER-229
> URL: https://issues.apache.org/jira/browse/MRESOLVER-229
> Project: Maven Resolver
> Issue Type: Improvement
> Components: Resolver
> Affects Versions: 1.7.1
> Reporter: wei cai
> Priority: Major
>
> We are tuning maven builds in our company, so far, we've made 2 big changes
> that we think they can benefit the opensource community.
> * maven-resolver change by a skip & reconcile approach
> https://issues.apache.org/jira/browse/MRESOLVER-228?filter=allopenissues
> * download poms and jars in parallel ahead as a warm-up
> It is not a new issue that downloading dependencies in maven is slow. Similar
> issues reported in:
> https://issues.apache.org/jira/browse/MRESOLVER-7
> Our solution is a bit different, basically we:
> * Collect artifacts required for the project by listening to the maven
> dependency resolve event and store as a artifacts.json (GAV list & its
> repository ids) somewhere, in our case, we are storing the artifacts.json in
> a backend service.
> * Before maven build kick starts, we called the maven-resolver api
> (DefaultDependencyCollector) to resolve all artifacts in artifacts.json in
> parallel.
> * After warm-up completed, let maven still build the project as normal. This
> means even something missed in artifacts.json, maven could still find the
> missing part and resolve them.
> This solution does not help for the very 1st build as no artifacts.json is
> built yet, however for any future builds, download can happen in parallel and
> thus it would be very quick.
> This solution helps speed up the maven builds in below cases:
> * All CI builds
> * New hires or new machines as there is no local maven repo cache
> * Snapshot updates
> * When a new dependency added, especially for a dependency having lots of
> transitive dependencies
> * Version upgrade such as Spring boot version upgrade
> [~michael-o]
> As a opensource solution, may be we can store the artifacts.json to .mvn in
> the git repository root, in this way, the artifacts list cache goes with the
> code.
> Please share your comments with this solution.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
