tarun11Mavani commented on code in PR #15863:
URL: https://github.com/apache/pinot/pull/15863#discussion_r2100290103
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskGenerator.java:
##########
@@ -214,15 +216,24 @@ public List<PinotTaskConfig>
generateTasks(List<TableConfig> tableConfigs) {
continue;
}
// TODO see if multiple groups of same partition can be added
- Map<String, String> configs = new
HashMap<>(getBaseTaskConfigs(tableConfig,
- groups.get(0).stream().map(x ->
x.getSegmentZKMetadata().getSegmentName()).collect(Collectors.toList())));
+
+ List<String> segmentNames =
+ groups.get(0).stream().map(x ->
x.getSegmentZKMetadata().getSegmentName()).collect(Collectors.toList());
+ //get max creation time for the segments across all servers. This will
be used as the creation time of the
+ // merge segment
+ Long maxCreationTimeMillis =
+ getMaxCreationTimeMillis(tableNameWithType, segmentNames,
serverToSegments, serverToEndpoints,
Review Comment:
Good catch.
As discussed offline, chances of this happening is quite low and if it
happens this just means that we will not gain any advantage from that
particular task run. In the next run, we will most likely pickup another set of
segments or another segment will be added in the same set, resulting in a
successful task run.
Maybe we could add some observability to track such behaviour?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
