[
https://issues.apache.org/jira/browse/SOLR-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186938#comment-17186938
]
Ilan Ginzburg commented on SOLR-14613:
--------------------------------------
I would definitely love to have more eyeballs on this
[PR1684|https://github.com/apache/lucene-solr/pull/1684].
To make it easier, a way to approach it is as follows:
[{{Assign.createAssignStrategy()}}|https://github.com/apache/lucene-solr/pull/1684/files#diff-3fd945fa4f7d44dfc97595494bb8ea18R552]
picks between existing placement strategies and the new plugin one.
[{{PlacementPluginAssignStrategy.assign()}}|https://github.com/apache/lucene-solr/pull/1684/files#diff-5d6e5294192d97f9b0a4d6baa22d25e2R56]
is the glue between existing Solr placement infra and the plugin code,
transforming Solr abstractions into the plugin abstractions and vice versa.
A sample initial (toy) plugin example can be found in
{{[SamplePluginMinimizeCores|https://github.com/apache/lucene-solr/pull/1684/files#diff-9e00a78f3810f5e5c3907723fa9d187c]}}.
It does implement the placement logic and computation (in a suboptimal way,
WIP) and shows how it interacts with the placement plugin interfaces defined in
this PR.
There are 5 families of interfaces that a plugin interacts with:
1. Observing existing cluster topology:
{{[Cluster|https://github.com/apache/lucene-solr/pull/1684/files#diff-b69b4e75815b3c67aae07fc92f91608d]}},
{{[Node|https://github.com/apache/lucene-solr/pull/1684/files#diff-9e130453afd9b39c1a421906e28a4c4a]}},
{{[SolrCollection|https://github.com/apache/lucene-solr/pull/1684/files#diff-9cb21e61da6a3b1e537de35b78eedb1a]}},
{{[Shard|https://github.com/apache/lucene-solr/pull/1684/files#diff-de4d5a286497ffa09c92901e5282fd6a]}},
{{[Replica|https://github.com/apache/lucene-solr/pull/1684/files#diff-b236471170ce1c259c1a3fbc0aeb5c13]}}.
2. Receiving a placement request, for example:
{{[AddReplicasPlacementRequest|https://github.com/apache/lucene-solr/pull/1684/files#diff-2066869526c1471e9ec46d71faec1826]}}
3. Preparing to fetch additional data from the cluster by creating
{{PropertyKey}} instances using
{{[PropertyKeyFactory|https://github.com/apache/lucene-solr/pull/1684/files#diff-5d65928113495143e29323a88ee6bf9f]}}
4. Fetching the additional data using
{{[PropertyValueFetcher|https://github.com/apache/lucene-solr/pull/1684/files#diff-a55062703fa3cf875dea2542e930e82e]}}
and getting the corresponding
{{[PropertyValue|https://github.com/apache/lucene-solr/pull/1684/files#diff-d4ace968d26586330e5b20a0b698e9a7]}}
(for example
{{[CoresCountPropertyValue|https://github.com/apache/lucene-solr/pull/1684/files#diff-955a2dda6d825451ceb86a5b4e452d3b]}})
5. Creating the placement decisions and returning them as plugin output, using
{{[PlacementPlanFactory|https://github.com/apache/lucene-solr/pull/1684/files#diff-f416298dbc7b2fc4a569737da5174fc5]}}
> Provide a clean API for pluggable replica assignment implementations
> --------------------------------------------------------------------
>
> Key: SOLR-14613
> URL: https://issues.apache.org/jira/browse/SOLR-14613
> Project: Solr
> Issue Type: Improvement
> Components: AutoScaling
> Reporter: Andrzej Bialecki
> Assignee: Ilan Ginzburg
> Priority: Major
> Time Spent: 20h 40m
> Remaining Estimate: 0h
>
> As described in SIP-8 the current autoscaling Policy implementation has
> several limitations that make it difficult to use for very large clusters and
> very large collections. SIP-8 also mentions the possible migration path by
> providing alternative implementations of the placement strategies that are
> less complex but more efficient in these very large environments.
> We should review the existing APIs that the current autoscaling engine uses
> ({{SolrCloudManager}} , {{AssignStrategy}} , {{Suggester}} and related
> interfaces) to see if they provide a sufficient and minimal API for plugging
> in alternative autoscaling placement strategies, and if necessary refactor
> the existing APIs.
> Since these APIs are internal it should be possible to do this without
> breaking back-compat.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
