Re: Modularizing Solr with new contrib packages

Thanks, Jan for all the explanations and sorry for the off-topic intrusion on the other thread. All makes sense, Cheers -- Alessandro Benedetti Apache Lucene/Solr PMC member and Committer Director, R&D Software Engineer, Search Consultant www.sease.io On Fri, 21 Jan 2022

Re: Modularizing Solr with new contrib packages

Yeah +1 to increase modularization in general. If for some reason this makes the functionality harder to use (which I sympathize with), I think we should instead direct our energy to making modules/packages easier to use. I'm thrilled about Jan's proposal to simply list the module names on bin/sol

Re: Modularizing Solr with new contrib packages

There was a message from Alessandro in another thread , which I'll answer here instead: (I'm posting the full question below my answers here for context) Many questions, and definitely a tangent :) But let me try a short answer >

Re: Modularizing Solr with new contrib packages

+1 to the developer-doc, very neat. Thanks for adding it. On Sun, Jan 16, 2022 at 10:52 PM Eric Pugh wrote: > I found this a great summary! > > On Jan 16, 2022, at 9:48 AM, Jan Høydahl wrote: > > I added a developer-doc draft for modules and packages in > https://github.com/apache/solr/pull/531

Re: Modularizing Solr with new contrib packages

I found this a great summary! > On Jan 16, 2022, at 9:48 AM, Jan Høydahl wrote: > > I added a developer-doc draft for modules and packages in > https://github.com/apache/solr/pull/531 > (HTML preview >

Re: Modularizing Solr with new contrib packages

I added a developer-doc draft for modules and packages in https://github.com/apache/solr/pull/531 (HTML preview ). Let me know if it is useful. Jan > 14. jan. 2022 kl. 18:13 skr

Re: Modularizing Solr with new contrib packages

I wish to revive SOLR-14688, and make sure your immutable deployments are honoured for first party packages. I'll take a stab at it over the weekend. On Fri, 14 Jan, 2022, 10:44 pm David Smiley, wrote: > Fair points. I might take a stab at this on the weekend to see. > > I propose no change to

Re: Modularizing Solr with new contrib packages

Fair points. I might take a stab at this on the weekend to see. I propose no change to the SOLR_HOME detection logic, which will naturally end up being SOLR_INSTALL/server/solr (where solr.xml is). Docker stuff won't need to set it / play games as it does now. ~ David Smiley Apache Lucene/Solr

Re: Modularizing Solr with new contrib packages

Hmm, yea it's always been a bit odd how SOLR_HOME does not point to where you untared solr, i.e. /opt/solr, like for every other software out there. So I support such a change. Will SOLR_VAR be exactly what the old SOLR_HOME was, i.e. /var/solr/data, or will it point to /var/solr? It's also a bi

Re: Modularizing Solr with new contrib packages

I believe the root cause here is fixed by my "Immutable Infrastructure" adherence proposal relating to a new SOLR_VAR: https://lists.apache.org/thread/3vvld3xnndtthtl7sfgdbsgkbtpm55b0 Thus SOLR_HOME stays with the solr installation; mutable data like the indexes go in a new SOLR_VAR -- ultimately t

Re: Modularizing Solr with new contrib packages

Yep, have also been using SOLR_HOME/lib for years. But for a recent client, they needed to package up 2-3 plugin jars into the docker image, so then we tried $SOLR_HOME/lib, but since /var/solr/data is defined as a Docker volume in our Dockerfile, it won't help copying libs in that location in c

Re: Modularizing Solr with new contrib packages

+1 to your phasing. > Another minor improvement for users is if we pre-add $SOLR_TIP/lib to the > classloader I'll create a JIRA :) SOLR-HOME/lib is already supported -- https://nightlies.apache.org/solr/draft-guides/solr-reference-guide-main/libs.html This is what I recommend people use in ge

Re: Modularizing Solr with new contrib packages

> > It could very well be worth shipping two docker images in the meantime. > Or maybe a zip of each module could be a separate artifact that is > published? I'm not sure what freedoms we have to do this in the ASF. > I think for 9.0 we could realistically shoot for 2 binary releases and 2 docker

Re: Modularizing Solr with new contrib packages

Another minor improvement for users is if we pre-add $SOLR_TIP/lib to the classloader, similar to what we have with $SOLR_HOME/lib today. The disadvantage of $SOLR_HOME/lib is that it can be anywhere, perhaps on a Docker volume or a different disk, so you cannot e.g make a Dockerfile like FROM

Re: Modularizing Solr with new contrib packages

There is not a lack of vision for future local and remote package repositories, but the story is that package mgmt development has stalled, and is out of reach for 1st party pkgs in the 9.0.0 timeframe. So we have to think progress over perfection - once again Phase 1. (9.0): Modularize Solr by

Re: Modularizing Solr with new contrib packages

Shawn: * RE redundancies of stuff in /dist/, see https://issues.apache.org/jira/browse/SOLR-15916 * RE "contrib" vs "module" vs "package", see: https://issues.apache.org/jira/browse/SOLR-15917 * RE not shipping these extras with the Solr distribution, see: "slim distro" mention in the document "Sol

Re: Modularizing Solr with new contrib packages

On 1/12/2022 8:31 AM, Jan Høydahl wrote: I think there are lots of pieces of code in solr-core that can easily be extracted the same way. Some perhaps even for 9.0.0, as it slims down the core and reduces attack surface for most users as well. I think it would be really awesome if we had a co

Re: Modularizing Solr with new contrib packages

(my previous email was accidentally sent; it was incomplete!) Speaking of modularization: * https://issues.apache.org/jira/browse/SOLR-15904 "Move SQLHandler to a contrib/module/package" -- just a JIRA issue; I don't have time for this one now. * https://issues.apache.org/jira/browse/SOLR-14660 "M

Re: Modularizing Solr with new contrib packages

All awesomeness! Speaking of modularization: * https://issues.apache.org/jira/browse/SOLR-15904 Move SQLHandler to a contrib/module/package -- just a JIRA issue; I don't have time for this one now. * https://issues.apache.org/jira/browse/SOLR-14660 ~ David Smiley Apache Lucene/Solr Search Develo

Modularizing Solr with new contrib packages

Hi, I just did an attempt to lift out the JWT auth plugin from solr-core into its own contrib [1] and it wasn't too hard. I think it gives much better insight into the dependency situation and nice to have a separate solr-jwt-auth-9.0.0.jar This is also a first step towards converting it to a p