tarun11Mavani commented on code in PR #17126:
URL: https://github.com/apache/pinot/pull/17126#discussion_r2489669393
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompaction/UpsertCompactionTaskGenerator.java:
##########
@@ -269,8 +269,17 @@ public static SegmentSelectionResult
processValidDocIdsMetadata(Map<String, Stri
return 1;
});
- return new SegmentSelectionResult(
-
segmentsForCompaction.stream().map(Map.Entry::getKey).collect(Collectors.toList()),
segmentsForDeletion);
+ // Apply randomization if configured
+ List<SegmentZKMetadata> finalSegmentsForCompaction;
+ double randomizationFactor = Double.parseDouble(
+
taskConfigs.getOrDefault(UpsertCompactionTask.SEGMENT_SELECTION_RANDOMIZATION_FACTOR,
+
String.valueOf(UpsertCompactionTask.DEFAULT_SEGMENT_SELECTION_RANDOMIZATION_FACTOR)));
+
+ // Apply segment selection with randomization using BaseTaskGenerator
utility
+ finalSegmentsForCompaction =
BaseTaskGenerator.selectRandomItems(segmentsForCompaction, maxTasks,
+ randomizationFactor);
Review Comment:
finalSegmentsForCompaction is correctly typed as List<SegmentZKMetadata>,
which is exactly what the SegmentSelectionResult constructor expects
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
