ege-st commented on code in PR #12334:
URL: https://github.com/apache/pinot/pull/12334#discussion_r1470236803
##########
pinot-controller/src/main/java/org/apache/pinot/controller/util/ServerSegmentMetadataReader.java:
##########
@@ -140,16 +143,17 @@ public TableMetadataInfo
getAggregatedTableMetadataFromServer(String tableNameWi
return v;
});
- // Since table segments may have multiple replicas, divide
diskSizeInBytes, numRows and numSegments by numReplica
- // to avoid double counting, for columnAvgLengthMap,
columnAvgCardinalityMap and maxNumMultiValuesMap, dividing by
- // numReplica is not needed since totalNumSegments already contains
replicas.
+ // Since table segments may have multiple replicas, divide
diskSizeInBytes, numRows, numSegments and primary key
+ // count by numReplica to avoid double counting, for columnAvgLengthMap,
columnAvgCardinalityMap and
+ // maxNumMultiValuesMap, dividing by numReplica is not needed since
totalNumSegments already contains replicas.
totalDiskSizeInBytes /= numReplica;
totalNumSegments /= numReplica;
totalNumRows /= numReplica;
+ upsertPartitionToPrimaryKeyCountMap.replaceAll((k, v) -> v / numReplica);
Review Comment:
I think (and the preceding calculations for totalDiskSizeInBytes, etc.) this
assumes that each replica will have the exact same number of PKs as the other
replicas, but what happens if that is not the case (ie, one replica is
inconsistent and has a different number of PKs)? That discrepancy will get lost
here but could be very valuable to have.
Is it possible for `upsertPartitionToPrimaryKeyCountMap` to store PK count
for each Replica of each partition/segment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
