Jackie-Jiang commented on PR #9199: URL: https://github.com/apache/pinot/pull/9199#issuecomment-1215987719
> > This has the same behavior as setting `numInstancesPerPartition` to 1, and `numPartitions` to the partitions for the table. Currently we don't support explicitly setting partitions for `RealtimeSegmentAssignment`, so we should fix that instead of introducing this new strategy > > I feel this is not exactly the same as `numInstancesPerPartition=1`, instead `numInstancesPerPartition` shall be the same as the number of replicas. In fact, the strategy is somewhat deterministic as the added stragy name suggests. And the main purpose is to make the assignment declarative and reduce the operational cost, e.g. we dont need to update `numInstancesPerPartition` when we increase `numPartitions` to scale out the table etc This new strategy is trying to put all segments from the same segment partition to the same server, which is exactly the same as what `numInstancesPerPartition = 1` is doing (and it is also deterministic). In order to achieve this, you should use the partition based replica-group assignment, and always put `numInstancesPerPartition = 1`. You may increase `numPartitions` if needed (should be very rare because it should be the same as table partitions, and changing it requires re-partitioning all the segments). Noted that `numPartitions` can be larger than the total instances within a replica-group. E.g. with 9 servers, you can have 3 replica-groups (3 servers per replica), then 18 partitions, and each server will host 6 partitions. You may scale it up to 18 servers, and run a rebalance without changing `numInstancesPerPartition` or `numPartitions`. After rebalancing, each server will host 3 partitions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
