Ok, get the meaning of preferences. Would there be a way to write a generic rule that would suggest moving shards to obtain balance, without specifying absolute core counts? I.e. if you have three nodes A: 3 cores B: 5 cores C: 3 cores
Then that rule would suggest two moves to end up with 4 cores on all three (unless that would violate disk space or load limits)? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 12. jun. 2018 kl. 08:10 skrev Shalin Shekhar Mangar <shalinman...@gmail.com>: > > Hi Jan, > > Comments inline: > > On Tue, Jun 12, 2018 at 2:19 AM Jan Høydahl <jan....@cominvent.com > <mailto:jan....@cominvent.com>> wrote: > >> Hi >> >> I'm trying to have Autoscaling move a shard to another node after manually >> splitting. >> We have two nodes, one has a shard1 and the other node is empty. >> >> After SPLITSHARD you have >> >> * shard1 (inactive) >> * shard1_0 >> * shard1_1 >> >> For autoscaling we have the {"minimize" : "cores"} cluster preference >> active. Because of that I'd expect that Autoscaling would suggest to move >> e.g. shard1_1 to the other (empty) node, but it doesn't. Then I create a >> rule just to test {"cores": "<2", "node": "#ANY"}, but still no >> suggestions. Not until I delete the inactive shard1, then it suggests to >> move one of the two remaining shards to the other node. >> >> So my two questions are >> 1. Is it by design that inactive shards "count" wrt #cores? >> I understand that it consumes disk but it is not active otherwise, >> so one could argue that it should not be counted in core/replica rules? >> > > Today, inactive slices also count towards the number of cores -- though > technically correct, it is probably an oversight. > > >> 2. Why is there no suggestion to move a shard due to the "minimize cores" >> reference itself? >> > > The /autoscaling/suggestions end point only suggests if there are policy > violations. Preferences such as minimize:cores are more of a sorting order > so they aren't really being violated. After you add the rule, the framework > still cannot give a suggestion that satisfies your rule. This is because > even if shard1_1 is moved to node2, node1 still has shard1 and shard1_0. So > the system ends up not suggesting anything. You should get a suggestion if > you add a third node to the cluster though. > > Also see SOLR-11997 <https://issues.apache.org/jira/browse/SOLR-11997 > <https://issues.apache.org/jira/browse/SOLR-11997>> which > will tell users that a suggestion could not be returned because we cannot > satisfy the policy. There are a slew of other improvements to suggestions > planned that will return suggestions even when there are no policy > violations. > > >> >> -- >> Jan Høydahl, search solution architect >> Cominvent AS - www.cominvent.com <http://www.cominvent.com/> >> >> > > -- > Regards, > Shalin Shekhar Mangar.
