[
https://issues.apache.org/jira/browse/SOLR-14688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17246314#comment-17246314
]
Tomas Eduardo Fernandez Lobbe commented on SOLR-14688:
------------------------------------------------------
There were some discussions in Slack the last couple days that I'd like to
bring here since they are related to this Jira issue. The threads are
[this|https://the-asf.slack.com/archives/CNMTSU970/p1582794103004800] and
[this|https://the-asf.slack.com/archives/CNMTSU970/p1607019429038000]. While
there are different opinions expressed in those threads, the general sentiment
is that there are missing pieces to the stories of packages/plugins, that need
to be resolved before we proceed here. In particular, compatibility and offline
installs. I'm now going to bring a summary of my comments and concerns here.
In the current state of things with package manager and if this issue is
implemented, someone could technically install a package from a different
version, but this can only be achieved by ensuring binary compatibility between
core and the packages across those different versions. This is way more than
what we guarantee today (think of something in analysis-extras calling
{{coreContainer.getCore().getLatestSchema().callSomethingAddedIn8.9()}}, would
break in any core version previous to 8.9). Guaranteeing this binary
compatibility would require a ton of testing (every package version against
every core version that's supported), it would put a lot of burden on us
developers, making it very difficult to add/change/deprecate/retire code and
may even make major upgrades impossible, or if we just take binary
compatibility as a "best effort", we'll make it difficult on the users to
figure out which version of what is compatible with other versions.
One question that I raised is, why do we want people to install a newer
contribs/packages into an older core version? Why don't we instead encourage
people to upgrade Solr by making it easier to do? Major version upgrades could
be more problematic because of index compatibility, yes, but really, having
binary compatibility across major upgrades is going to be very, very hard.
There is also great concern about the inability to install packages offline,
and how that affects the ability to install/deploy first/third party plugins (a
bunch of people expressed this in particular in those Slack threads I
mentioned). I believe the root of the problem is the fact that packages *have*
to be cluster-wide now. Instead of being able to create the deployable in some
build infrastructure, away from production environments, and then move that
deployable across your different environments such as "dev", "qa", "prod", or
whatever you have, the current implementation only allows one to configure a
cluster once it's created and running, doing API calls (forcing to enable
package manager AFAIK, even if no code needs to be added dynamically later),
and exposing the production environment to either a package repository or even
internet.
I believe packages (first, or third party) could work better if they could be
local to a node (and this doesn't mean there can't be cluster-wide packages,
but we need at least the "local" option). People could then, for example,
create their Docker image like (and these are not real commands, just get the
idea):
{noformat}
FROM official-docker-image-slim:x.y.z
ADD /some/build/path/custom-plugin1 /some/location/in/solr/custom-plugin1
RUN /solr/bin/solr install custom-plugin1 /some/location/in/solr/custom-plugin1
RUN /solr/bin/solr install analysis-extra
solr.apache.org/packages/analysis-extra/x.y.z
#or RUN /solr/bin/solr install analysis-extra /first/party/plugins/location
{noformat}
(The example is with Docker, but similar things can be done with other
deployables, like AMIs in AWS, or I'm sure any container technology.)
And then just build it and deploy it. If you are using the Kubernetes Solr
operator, it's a single command and the upgrade will start safely and
automatically. It's also important to mention that any upgrade could look just
the same, regardless if what you changed was Solr core, first party or third
party plugin.
I'm +1 on making the code more modular and independent, have better, well
thought interfaces like the one created for the replica placement framework and
much of the work ab has been doing to define higher level interfaces of things
currently need rework, but I think with the current state of package manager,
with cluster-wide packages, this issue is very dangerous.
> First party package implementation design
> -----------------------------------------
>
> Key: SOLR-14688
> URL: https://issues.apache.org/jira/browse/SOLR-14688
> Project: Solr
> Issue Type: Improvement
> Reporter: Noble Paul
> Priority: Major
> Labels: package, packagemanager
>
> Here's the design document for first party packages:
> https://docs.google.com/document/d/1n7gB2JAdZhlJKFrCd4Txcw4HDkdk7hlULyAZBS-wXrE/edit?usp=sharing
> Put differently, this is about package-ifying our "contribs".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
