[
https://issues.apache.org/jira/browse/MRESOLVER-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Cservenak updated MRESOLVER-346:
--------------------------------------
Description:
The locking that is present in Resolver since 1.8.0 is too eager:
* there are no shared locks used at all
* this implies that local repository access is heavily serialized by locking
* there is no "upgrade" of locking due that above
* consequence is that "hot" artifacts in bigger multi module build run in
parallel become bottleneck as all threads will wait for their moment to grab
exclusive lock.
We can do it better: there are 4 main areas where locking is used:
* ArtifactInstaller: it is about to install (write) artifact files to local
repository, it needs exclusive lock, *no change needed*.
* ArtifactDeployer: it is about to upload present files to remote, it does not
modifies anything on local. *Change it's lock to shared*. The exclusive lock
also meant that if no DeployAtEnd used, other modules during resolution had to
wait while this module uploaded.
* ArtifactResolver and MetadataResolver: two very similar beasts, they attempt
to resolve locally (from local repo) w/o any content modification (read only),
and if not successful, they will reach remote to download what is needed
(write). Here we *could do something similar to
[DCL|https://en.wikipedia.org/wiki/Double-checked_locking] is*: try with shared
lock first, and if local content is not fulfilling, release shared, acquire
exclusive and REDO all (as meanwhile someone else may downloaded files already).
was:
The locking that is present in Resolver since 1.8.0 is too eager:
* there are no shared locks used at all
* this implies that local repository access is heavily serialized by locking
* there is no "upgrade" of locking due that above
* consequence is that "hot" artifacts in bigger multi module build run in
parallel become bottleneck as all threads will wait for their moment to grab
exclusive lock.
We can do it better: there are 4 main areas where locking is used:
* ArtifactInstaller: it is about to install (write) artifact files to local
repository, it needs exclusive lock, no change needed.
* ArtifactDeployer: it is about to upload present files to remote, it does not
modifies anything on local. Change it's lock to shared. The exclusive lock also
meant that if no DeployAtEnd used, other modules during resolution had to wait
while this module uploaded.
* ArtifactResolver and MetadataResolver: two very similar beasts, they attempt
to resolve locally (from local repo) w/o any content modification (read only),
and if not successful, they will reach remote to download what is needed
(write). Here we could do something similar to
[DCL|https://en.wikipedia.org/wiki/Double-checked_locking] is: try with shared
lock first, and if local content is not fulfilling, release shared, acquire
exclusive and REDO all (as meanwhile someone else may downloaded files already).
> Too eager locking
> -----------------
>
> Key: MRESOLVER-346
> URL: https://issues.apache.org/jira/browse/MRESOLVER-346
> Project: Maven Resolver
> Issue Type: Bug
> Components: Resolver
> Reporter: Tamas Cservenak
> Priority: Major
> Fix For: 1.9.8
>
>
> The locking that is present in Resolver since 1.8.0 is too eager:
> * there are no shared locks used at all
> * this implies that local repository access is heavily serialized by locking
> * there is no "upgrade" of locking due that above
> * consequence is that "hot" artifacts in bigger multi module build run in
> parallel become bottleneck as all threads will wait for their moment to grab
> exclusive lock.
> We can do it better: there are 4 main areas where locking is used:
> * ArtifactInstaller: it is about to install (write) artifact files to local
> repository, it needs exclusive lock, *no change needed*.
> * ArtifactDeployer: it is about to upload present files to remote, it does
> not modifies anything on local. *Change it's lock to shared*. The exclusive
> lock also meant that if no DeployAtEnd used, other modules during resolution
> had to wait while this module uploaded.
> * ArtifactResolver and MetadataResolver: two very similar beasts, they
> attempt to resolve locally (from local repo) w/o any content modification
> (read only), and if not successful, they will reach remote to download what
> is needed (write). Here we *could do something similar to
> [DCL|https://en.wikipedia.org/wiki/Double-checked_locking] is*: try with
> shared lock first, and if local content is not fulfilling, release shared,
> acquire exclusive and REDO all (as meanwhile someone else may downloaded
> files already).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
