tarun11Mavani commented on code in PR #17126:
URL: https://github.com/apache/pinot/pull/17126#discussion_r2489654697
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompaction/UpsertCompactionTaskGenerator.java:
##########
@@ -269,8 +269,17 @@ public static SegmentSelectionResult
processValidDocIdsMetadata(Map<String, Stri
return 1;
});
- return new SegmentSelectionResult(
-
segmentsForCompaction.stream().map(Map.Entry::getKey).collect(Collectors.toList()),
segmentsForDeletion);
+ // Apply randomization if configured
+ List<SegmentZKMetadata> finalSegmentsForCompaction;
+ double randomizationFactor = Double.parseDouble(
+
taskConfigs.getOrDefault(UpsertCompactionTask.SEGMENT_SELECTION_RANDOMIZATION_FACTOR,
+
String.valueOf(UpsertCompactionTask.DEFAULT_SEGMENT_SELECTION_RANDOMIZATION_FACTOR)));
+
+ // Apply segment selection with randomization using BaseTaskGenerator
utility
+ finalSegmentsForCompaction =
BaseTaskGenerator.selectRandomItems(segmentsForCompaction, maxTasks,
+ randomizationFactor);
Review Comment:
segmentsForCompaction is declared as List<Pair<SegmentZKMetadata, Long>>,
which is exactly what selectRandomItems expects. The types are perfectly
compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
