Re: Moving out all Hadoop plugins into one module

2022-01-20 Thread Jan Høydahl
That's why a package system tends to be a complex beast, with a dependency tree between packages etc, so you'd have a hadoop-common package and hadoop-auth and hadoop-hdfs that depend on it. But I don't know if we want to go there, package management is not Solr's core business. Another thing t

Re: Moving out all Hadoop plugins into one module

2022-01-20 Thread Kevin Risden
Yea it would be duplicate jars in both places. It is a shame both share the name "hadoop" since the two features - filesystem and authentication. They end up being two entirely different things both in Hadoop itself and inside of Solr. Kevin Risden On Thu, Jan 20, 2022 at 4:58 PM David Smiley w

Re: Moving out all Hadoop plugins into one module

2022-01-20 Thread David Smiley
Separate modules will mean our distro will end up duplicating hadoop-common and other related JARs for both modules. I was trying to be practical. But it's not important to me; ok. implementation ('org.apache.hadoop:hadoop-common') { transitive = false } // too many to ignore implementation ('org

Re: Moving out all Hadoop plugins into one module

2022-01-20 Thread Kevin Risden
My preference would be as a separate HadoopAuthentication or something module. HDFS the filesystem / blockcache / etc support is unique and separate from the authentication part. It shouldn't all be in one module. Kevin Risden On Thu, Jan 20, 2022 at 4:48 PM David Smiley wrote: > The issue htt

Moving out all Hadoop plugins into one module

2022-01-20 Thread David Smiley
The issue https://issues.apache.org/jira/browse/SOLR-14660 is about moving the HDFS plugins out of core into a module. While a great thing, it still leaves quite a few Hadoop related dependencies in solr-core because Hadoop is not there only for HDFS; it's there for some exotic authentication & au