klsince commented on a change in pull request #7961:
URL: https://github.com/apache/pinot/pull/7961#discussion_r778974928
##########
File path:
pinot-server/src/main/java/org/apache/pinot/server/starter/helix/BaseServerStarter.java
##########
@@ -363,6 +377,41 @@ private void startupServiceStatusCheck(long endTimeMs) {
ServiceStatus.getStatusDescription());
}
+ /**
+ * Recursively deletes all data directories starting with tmp- last modified
before the startTime.
+ * @param startTime start time of the application
+ * @param dataDir data directory to start from
+ */
+ @VisibleForTesting
+ public static void deleteTempFilesSinceCutoffTime(long startTime, @Nonnull
File dataDir) {
+ if (!dataDir.exists() || !dataDir.isDirectory()) {
+ LOGGER.warn("Data directory {} does not exist or is not a directory",
dataDir);
+ return;
+ }
+ IOFileFilter beforeStartTimeFilter = new AgeFileFilter(startTime, true);
+ IOFileFilter tmpPrefixFilter = new PrefixFileFilter("tmp-");
Review comment:
Thanks for the proposals. I prefer to option 2 as well.
There are a few places creating tmp- dir in pinot/core/data/manager module.
Your util method may be used to unify all those cases. Right now, their dir
suffix are unique, which (luckily) could help one figure out where deletion
fails. So for the new util method, I'd suggest to allow caller to customize
some part of the dir name like its suffix to help debugging.
```
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
File tempRootDir = getSegmentDataDir("tmp-" + segmentName + "-" +
UUID.randomUUID());
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/HLRealtimeSegmentDataManager.java:
File tempSegmentFolder = new File(_resourceTmpDir, "tmp-" +
System.currentTimeMillis());
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java:
File tempSegmentFolder = new File(_resourceTmpDir, "tmp-" +
_segmentNameStr + "-" + now());
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java:
File tempSegmentDir = new File(_indexDir, "tmp-" + segmentName + "." +
System.currentTimeMillis());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
