[
https://issues.apache.org/jira/browse/SOLR-12730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki resolved SOLR-12730.
-------------------------------------
Fix Version/s: (was: master (9.0))
Resolution: Fixed
This has been fixed a long time ago (and subsequently removed from 9.0).
> Implement staggered SPLITSHARD requests in IndexSizeTrigger
> -----------------------------------------------------------
>
> Key: SOLR-12730
> URL: https://issues.apache.org/jira/browse/SOLR-12730
> Project: Solr
> Issue Type: Improvement
> Components: AutoScaling
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Fix For: 8.1
>
>
> Simulated large scale tests uncovered an interesting scenario that occurs
> also in real clusters where {{IndexSizeTrigger}} is used for controlling the
> maximum shard size.
> As index size grows and the number of shards grows, if document assignment is
> more or less even then at equal intervals (on a {{log2}} scale) there will be
> an avalanche of SPLITSHARD operations, because all shards will reach the
> critical size at approximately the same time.
> A hundred or more split shard operations running in parallel may severely
> affect the cluster performance.
> One possible approach to reduce the likelihood of this situation is to split
> shards not exactly in half but rather fudge the proportions around 60/40% in
> a random sequence, so that the resulting sub-sub-sub…shards would reach the
> thresholds at different times. This would require modifications to the
> SPLITSHARD command to allow this randomization.
> Another approach would be to simply limit the maximum number of parallel
> split shard operations. However, this would slow down the process of reaching
> the balance (increase lag) and possibly violate other operational constraints
> due to some shards waiting too long for the split and significantly exceeding
> their max size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
