Sigma-Ma opened a new pull request, #8539: URL: https://github.com/apache/hbase/pull/8539
JIRA: https://issues.apache.org/jira/browse/HBASE-30291 ## What changes were proposed in this pull request? This pull request changes `BackupHFileCleaner` to load the referenced HFile names once in `preClean()` and reuse the resulting snapshot for all `getDeletableFiles()` calls in the same cleaner cycle. The snapshot is published only after loading completes. If loading the references fails, the snapshot is marked unavailable and all files are kept for that cleaner cycle. The existing `previousCleaningCompletionTimestamp` protection is preserved to prevent recently archived HFiles from being deleted before the next snapshot refresh. Tests are added to verify that: - multiple `getDeletableFiles()` calls in one cleaner cycle load the references only once; - the snapshot is refreshed in the next cleaner cycle; - a failed refresh does not reuse a stale snapshot or make files deletable. ## Why are the changes needed? `BackupHFileCleaner#getDeletableFiles()` is invoked concurrently for each non-empty archive directory. It currently scans the backup system tables on every invocation to load the same set of backed-up tables and bulk-loaded HFile references. On clusters with many regions, this causes tens of thousands of redundant system-table scans in a single cleaner cycle. The resulting I/O load can prevent the cleaner from keeping up, causing archived HFiles to accumulate. Loading the references once per cleaner cycle removes these redundant scans without introducing a cross-cycle cache or changing the general `CleanerChore` framework. ## How was this patch tested? The following checks were run successfully: ```shell mvn -pl hbase-backup -am \ -Dtest=TestBackupHFileCleaner \ -Dsurefire.failIfNoSpecifiedTests=false \ -DskipITs \ test mvn -pl hbase-backup -DskipITs test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
