DieterDP-ng commented on code in PR #7582:
URL: https://github.com/apache/hbase/pull/7582#discussion_r2676601419
##########
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/FullTableBackupClient.java:
##########
@@ -181,6 +189,50 @@ public void execute() throws IOException {
// set overall backup status: complete. Here we make sure to complete
the backup.
// After this checkpoint, even if entering cancel process, will let the
backup finished
backupInfo.setState(BackupState.COMPLETE);
+
+ // Scan oldlogs for dead/decommissioned hosts and add their max WAL
timestamps
+ // to newTimestamps. This ensures subsequent incremental backups won't
try to back up
+ // WALs that are already covered by this full backup's snapshot.
+ Path walRootDir = CommonFSUtils.getWALRootDir(conf);
+ Path logDir = new Path(walRootDir, HConstants.HREGION_LOGDIR_NAME);
+ Path oldLogDir = new Path(walRootDir, HConstants.HREGION_OLDLOGDIR_NAME);
+ FileSystem fs = walRootDir.getFileSystem(conf);
+
+ List<FileStatus> allLogs = new ArrayList<>();
+ for (FileStatus hostLogDir : fs.listStatus(logDir)) {
+ String host =
BackupUtils.parseHostNameFromLogFile(hostLogDir.getPath());
+ if (host == null) {
+ continue;
+ }
+ allLogs.addAll(Arrays.asList(fs.listStatus(hostLogDir.getPath())));
+ }
+ allLogs.addAll(Arrays.asList(fs.listStatus(oldLogDir)));
Review Comment:
There's apparently a config key `hbase.separate.oldlogdir.by.regionserver`
that adds the server address as a subfolder, so this line will not work as
expected when that's set to `true`. (Solution is to also check files that are
one folder deeper.)
Question: when exactly (and by what) are WALs moved to the oldWals folder
(oldLogDir)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
