[
https://issues.apache.org/jira/browse/SOLR-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262615#comment-17262615
]
Andrzej Bialecki commented on SOLR-15055:
-----------------------------------------
So, let's take a step back, and start with use-cases and requirements:
* cross-collection joins is the main use-case because it benefits greatly from
co-location of replicas for the primary (search) and secondary (join)
collections.
* when creating the primary collection (or adding new replicas to it) the
placement plugin must take this into account and enforce the co-location of new
primary replicas if possible.
** in 8x the framework would automatically create secondary replicas as
necessary to satisfy this requirement (at the cost of code complexity).
** now at minimum the placement plugin should fail if it's not possible to
satisfy this constraint with the current cluster layout, because then some
primary replicas would have to use secondary replicas from other nodes and this
would greatly (and unevenly) increase the query latency.
** if the placement of the primary collection fails because this constraint
cannot be satisfied (there are too few secondary replicas) then the operator
must manually add as many secondary replicas as needed. This is the weakness of
this minimal approach - it would be better to somehow automate it to eliminate
the manual intervention.
* removal of secondary replicas should be prevented if they are actively used
under this constraint by any primary replicas on a specific node - allowing
this would cause performance issues as described above.
* removal of primary replicas should cause the secondary replicas to be removed
as well, if they are no longer in use on a specific node - but ensuring that at
least N replicas of the secondary collection remain (to prevent data loss).
> Re-implement 'withCollection' and 'maxShardsPerNode'
> ----------------------------------------------------
>
> Key: SOLR-15055
> URL: https://issues.apache.org/jira/browse/SOLR-15055
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Solr 8x replica placement provided two settings that are very useful in
> certain scenarios:
> * {{withCollection}} constraint specified that replicas should be placed on
> the same nodes where replicas of another collection are located. In the 8x
> implementation this was limited in practice to co-locating single-shard
> secondary collections used for joins or other lookups from the main
> collection (which could be multi-sharded).
> * {{maxShardsPerNode}} - this constraint specified the maximum number of
> replicas per shard that can be placed on the same node. In most scenarios
> this was set to 1 in order to ensure fault-tolerance (ie. at most 1 replica
> of any given shard would be placed on any given node). Changing this
> constraint to values > 1 would reduce fault-tolerance but may be desired in
> test setups or as a temporary relief measure.
>
> Both these constraints are collection-specific so they should be configured
> e.g. as collection properties.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
