Jackie-Jiang commented on code in PR #8667:
URL: https://github.com/apache/pinot/pull/8667#discussion_r869768587
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -3396,6 +3398,42 @@ private void updateSegmentLineageEntryToReverted(String
tableNameWithType, Segme
System.currentTimeMillis()));
}
+ private List<String> deleteOrUpdateSegmentLineageEntryToReverted(String
tableNameWithType,
+ SegmentLineage segmentLineage, String segmentLineageEntryId,
LineageEntry lineageEntry,
+ List<String> newSegments) {
+ // Check that all segments from 'segmentsFrom' are in ONLINE state in the
external view.
+ Set<String> onlineSegments =
getOnlineSegmentsFromExternalView(tableNameWithType);
+
Preconditions.checkArgument(onlineSegments.containsAll(lineageEntry.getSegmentsFrom()),
String.format(
+ "Failed to update the lineage to be 'REVERTED'. Not all segments from
'segmentFrom' are in ONLINE state "
+ + "in the external view. (tableName = '%s', segmentsFrom = '%s',
onlineSegments = '%s'", tableNameWithType,
+ lineageEntry.getSegmentsFrom(), onlineSegments));
+
+ // Delete or update segmentsTo of the entry to revert to handle the case
of rerunning the protocol:
+ // Initial state:
+ // Entry1: { segmentsFrom: [s1, s2], segmentsTo: [s3, s4], status:
IN_PROGRESS}
+ // 1. Rerunning the protocol with s4 and s5, s4 should not be deleted to
avoid race conditions of concurrent data
+ // pushes and deletions:
+ // Entry1: { segmentsFrom: [s1, s2], segmentsTo: [s3], status: REVERTED}
+ // Entry2: { segmentsFrom: [s1, s2], segmentsTo: [s4, s5], status:
IN_PROGRESS}
+ // 2. Rerunning the protocol with s3 and s4, we can simply remove the
'IN_PROGRESS' entry:
+ // Entry2: { segmentsFrom: [s1, s2], segmentsTo: [s3, s4], status:
IN_PROGRESS}
+ List<String> segmentsToForEntryToRevert = new
ArrayList<>(lineageEntry.getSegmentsTo());
+ if (newSegments != null) {
+ segmentsToForEntryToRevert.removeAll(newSegments);
+ }
+
+ if (segmentsToForEntryToRevert.isEmpty() && lineageEntry.getState() ==
LineageEntryState.IN_PROGRESS) {
Review Comment:
The `IN_PROGRESS` check is redundant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
