VenuReddy2103 commented on code in PR #6397:
URL: https://github.com/apache/hive/pull/6397#discussion_r3028401426
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/FSRemover.java:
##########
@@ -126,6 +129,24 @@ private List<Path> removeFiles(CleanupRequest cr)
deleted.add(dead);
}
}
+ removeDatabaseDirIfNecessary(db, cr, fs);
return deleted;
}
+
+ private void removeDatabaseDirIfNecessary(Database db, CleanupRequest cr,
FileSystem fs) throws IOException {
+ if (db != null || !cr.isSoftDelete()) {
+ return;
+ }
+ Path databasePath = new Path(cr.getLocation()).getParent();
+ for (FileStatus status : fs.listStatus(databasePath)) {
+ if (status.isDirectory() &&
+ status.getPath().getName().matches("(.*)" +
AcidConstants.SOFT_DELETE_TABLE_PATTERN)) {
Review Comment:
Have tried to be safe ensuring that we have atleast one softdelete table
directory still exists. In case if some other files are directly created under
managed database directory going forward.
Have modified it now to use FileUtils.isDirEmpty(fs, path) as you suggest.
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/entities/CompactionInfo.java:
##########
@@ -368,4 +370,12 @@ public void setWriteIds(boolean hasUncompactedAborts,
Set<Long> writeIds) {
public boolean isAbortedTxnCleanup() {
return type == CompactionType.ABORT_TXN_CLEANUP;
}
+
+ public boolean isSoftDelete() {
+ return
"true".equalsIgnoreCase(getProperty(hive_metastoreConstants.SOFT_DELETE_DATABASE));
Review Comment:
done. Have modified this method for operation comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
