Hi Christos,
CCing dev@solr so that it stays in the archives.
I wanted to reach out to you to ask how you would resolve dependency
> conflicts with the carrotsearch dependency checks plugin. I am using the
> same dependencies.gradle file as in the Lucene project in Solr and I have
> found a way to "resolve" them, but I am not sure if that's the intended way.
>
Ok, so perhaps I should clarify that one of the objectives of removing the
consistent-versions plugin was to rely on gradle's built-in mechanisms. The
plugin you can see in Lucene (com.carrotsearch.gradle.dependencychecks) is
passive and only there to signal problems with conflicting versions in a
set of configurations we want to be "consistent". Any conflict dependency
or alignment should be done using gradle's mechanisms. I should also
mention that gradle's "conflicts" are a bit different to what I consider a
"conflict". For example, consider two subprojects A and B, where A depends
on library foo, B depends on library foo and baz:
A: foo
B: foo, baz
If you declare a version of foo in your catalog (say, 1.0) and baz has a
transitive dependency on foo 1.1, gradle will automatically bump foo to 1.1
in project B. So A will have foo 1.0 and B will have foo 1.1. This is how
gradle handles version resolution by default. There are various ways to
"align" those dependencies but I find all of them confusing (platforms,
etc.), sorry. Maybe you'll have more luck in applying those. :)
So, back to the main topic, the reason that custom plugin of mine is in
Lucene, is to detect situations when the same artifact exists in different
versions in a set of configurations of different subprojects. When such a
situation happens, it'll abort the build but won't do anything
automatically. How to resolve it is up to you. I opted to cross-check these:
def mainConfigurations = project.configurations.matching {
it.name in [
"compileClasspath",
"runtimeClasspath"
]
}
def testConfigurations = project.configurations.matching {
it.name in [
"annotationProcessor",
"testCompileClasspath",
"testRuntimeClasspath"
]
}
1. Add the conflicting dependency to the versions catalog (if not present)
> with the highest version from the conflict report
>
This is one possibility, yes. I always found it problematic to keep such
manually-tuned rules consistent in the presence of independent library
upgrades.
> 2. Force the dependency to the configuration group in
> the dependencies.gradle file via
> allprojects.dependencies.constraints.[configurationSet].configureEach {
> Configuration conf ->
> conf.resolutionStrategy {
> force [conflicting dependency from catalog]
> }
> }
>
I think this one is better. You can also add constraints like so
('consolidatedConfigurations' is a set of configurations we want to apply
the constraints to - it's typically the same :
def consolidatedConfigurations = ["compileClasspath", "runtimeClasspath",
(...and more) ]
dependencies {
constraints {
consolidatedConfigurations.configureEach { Configuration conf ->
add(conf.name, deps.log4j, {
because("Use consistent log4j version.")
})
add(conf.name, deps.junit, {
because("Use consistent JUnit version.")
})
}
}
}
these constraints are hints to gradle's version resolution [1]. It's still
a pain to figure out when such a constraint becomes unnecessary (for
example, because a dependency upgraded its transitive dependency or
something else has changed). I haven't found a way to detect unused
constraints automatically.
> I would go through the carrotsearch plugin and read the docs, but I
> couldn't find any. Do you maybe have a guide or something that I could use
> for reference?
>
No, sorry for this. The source code is your guide. :) It's really a simple
collection of dependencies from multiple configurations, then detecting
whether the same artifact has multiple versions and failing the build if
this happens. If all dependencies are consistent, they're checked against
versions.lock - if there are any differences, it means something has
changed (a direct or transitive dependency version has changed or has been
removed/ added). Such changes are reported and fail the build but this is
done for informational/ sanity reasons. I just don't trust gradle's
resolution mechanisms and want to make sure it works the way I want it to
work... :)
You can remove this plugin entirely if you want to - everything else should
work the same. I think it's good to have a plain-text list of all
dependencies in configurations we care about though -
"configurationGroups" : {
"main_dependencies" : {
"com.carrotsearch.randomizedtesting:randomizedtesting-runner:2.8.1" :
"fa9ef26b,refs=4",
"com.ibm.icu:icu4j:74.2" : "47ea4550,refs=6",
"commons-codec:commons-codec:1.13" : "e9962aab,refs=4",
"io.sgr:s2-geometry-library-java:1.0.0" : "cbc357ab,refs=4",
...
I would also recommend that you take a look at the Lucene commits that were
part of the patch that removed palantir's consistent versions plugin -
there were some quirks and rough edges that you'll see in the diffs. I'm
sure it'll be a similar situation in Solr.
If you can't figure out something, send an email to dev and CC me. I am by
no means a gradle expert but maybe I've come across something that I'll be
able to explain.
Good luck!
Dawid
[1]
https://docs.gradle.org/current/userguide/dependency_constraints.html#sec:adding-constraints-transitive-deps
>
> Sincerely,
> Christos Malliaridis
>
