somandal opened a new pull request, #15822: URL: https://github.com/apache/pinot/pull/15822
The rebalance batching needs to fetch the partitionId of the segments for `strictReplicaGroup` instance selector enabled tables. The logic added today blindly uses the CONSUMING segment logic for REALTIME tables that are using the `RealtimeSegmentAssignment` class for segment assignment. For scenarios when a partitionId is not found from segmentName or segment ZK metadata, this allocates a default partitionId as `segmentName.hashCode() % 10000`. For COMPLETED segments, the `RealtimeSegmentAssignment` actually used a different mechanism to assign a default partitionId if `partitionColumn == null` or `numPartitions = 1` for the COMPLETED instance partitions, which is to always allocate partitionId = 0. It falls back to the CONSUMING segment mechanism for other cases. This PR exposes to the partition ID fetcher details about whether the segment is CONSUMING or not, and utilized the instance partitions map to identify if a COMPLETED instance partitions exists for this table or not and makes decisions on partitionId calculations based on that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
