swaminathanmanish commented on code in PR #15048:
URL: https://github.com/apache/pinot/pull/15048#discussion_r1954585125
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -235,7 +236,11 @@ protected void removeSegmentFromStore(String
tableNameWithType, String segmentId
long retentionMs = deletedSegmentsRetentionMs == null
? _defaultDeletedSegmentsRetentionMs : deletedSegmentsRetentionMs;
String rawTableName =
TableNameBuilder.extractRawTableName(tableNameWithType);
- URI fileToDeleteURI = URIUtils.getUri(_dataDir, rawTableName,
URIUtils.encode(segmentId));
+ URI fileToDeleteURI = getFileToDeleteURI(rawTableName, segmentId);
+ if (fileToDeleteURI == null) {
+ LOGGER.warn("No segment file found for segment ID: {} in table: {}",
segmentId, rawTableName);
Review Comment:
Modify log to "No segment file for table, found in deep store" .
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -282,6 +287,41 @@ protected void removeSegmentFromStore(String
tableNameWithType, String segmentId
}
}
+ /**
+ * Gets URI for segment deletion by attempting to locate the segment file in
the following order:
+ * 1. Plain segment file (without extension)
+ * 2. Compressed segment file (with .tar.gz extension)
+ *
+ * @param rawTableName The name of the table
+ * @param segmentId The ID of the segment to delete
+ * @return URI of the existing segment file, or null if neither variant
exists or in case of filesystem errors
+ */
+ private URI getFileToDeleteURI(String rawTableName, String segmentId) {
+ try {
+ // Create both URIs upfront
+ URI plainFileUri = URIUtils.getUri(_dataDir, rawTableName,
URIUtils.encode(segmentId));
+ URI compressedFileUri = URIUtils.getUri(_dataDir, rawTableName,
+ URIUtils.encode(segmentId +
TarCompressionUtils.TAR_GZ_FILE_EXTENSION));
+
+ // Get filesystem instance once
+ PinotFS pinotFS = PinotFSFactory.create(plainFileUri.getScheme());
Review Comment:
Minor optimization - You can create pinotFS outside the method and pass it
to this method, instead of creating it here. This is outside in the caller as
well.
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -282,6 +287,41 @@ protected void removeSegmentFromStore(String
tableNameWithType, String segmentId
}
}
+ /**
+ * Gets URI for segment deletion by attempting to locate the segment file in
the following order:
+ * 1. Plain segment file (without extension)
+ * 2. Compressed segment file (with .tar.gz extension)
+ *
+ * @param rawTableName The name of the table
+ * @param segmentId The ID of the segment to delete
+ * @return URI of the existing segment file, or null if neither variant
exists or in case of filesystem errors
+ */
+ private URI getFileToDeleteURI(String rawTableName, String segmentId) {
+ try {
+ // Create both URIs upfront
+ URI plainFileUri = URIUtils.getUri(_dataDir, rawTableName,
URIUtils.encode(segmentId));
+ URI compressedFileUri = URIUtils.getUri(_dataDir, rawTableName,
+ URIUtils.encode(segmentId +
TarCompressionUtils.TAR_GZ_FILE_EXTENSION));
+
+ // Get filesystem instance once
+ PinotFS pinotFS = PinotFSFactory.create(plainFileUri.getScheme());
+
+ // Check for plain segment file first
+ if (pinotFS.exists(plainFileUri)) {
+ return plainFileUri;
+ }
+
+ // Check for compressed segment file
+ if (pinotFS.exists(compressedFileUri)) {
+ return compressedFileUri;
+ }
+
+ return null;
+ } catch (Exception e) {
+ return null;
Review Comment:
Can you add log.err here so that we know deep store lookup failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
