Jackie-Jiang commented on code in PR #16810:
URL: https://github.com/apache/pinot/pull/16810#discussion_r2430160467
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -1650,4 +1650,11 @@ private static void
closeSegmentDirectoryQuietly(SegmentDirectory segmentDirecto
}
}
}
+
+ /**
+ * Returns the configured peer download scheme if peer-to-peer download is
enabled; otherwise null.
+ */
+ public String getPeerDownloadScheme() {
Review Comment:
Annotate it with `@Nullable`
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java:
##########
@@ -583,6 +583,21 @@ public File downloadSegment(SegmentZKMetadata zkMetadata)
if (status.isCompleted()) {
// Segment is completed and ready to be downloaded either from deep
storage or from a peer (if peer-to-peer
// download is enabled).
+ String downloadUrl = zkMetadata.getDownloadUrl();
+ if (StringUtils.isNotEmpty(downloadUrl)) {
+ return super.downloadSegment(zkMetadata);
+ }
+ // Download URL might be missing due to race conditions or controller
commit failure. If peer download
+ // is enabled, try to fetch from ONLINE peers instead of failing fast.
+ if (_peerDownloadScheme != null) {
+ try {
+ return downloadSegmentFromPeers(zkMetadata);
+ } catch (Exception e) {
+ _logger.warn("Peer download attempt failed for completed segment:
{}. Falling back to deep store if possible",
+ zkMetadata.getSegmentName(), e);
+ }
+ }
+ // Fall back to the default behavior which will validate and potentially
throw if URL is still missing.
Review Comment:
This part shouldn't be required. The base class can handle completed segment
download
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
