[
https://issues.apache.org/jira/browse/HBASE-29572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-29572 started by huginn.
--------------------------------------
> Clean up output directory when copying the manifest fails during snapshot
> export
> --------------------------------------------------------------------------------
>
> Key: HBASE-29572
> URL: https://issues.apache.org/jira/browse/HBASE-29572
> Project: HBase
> Issue Type: Bug
> Components: snapshots
> Affects Versions: 2.4.11
> Reporter: huginn
> Assignee: huginn
> Priority: Minor
> Labels: pull-request-available
>
> If an export snapshot fails in copying .snapshotInfo and manifest, which may
> leave behind an empty snapshot directory under .snapshot
> (snapshot.export.skip.tmp = true) or .snapshot/.tmp (snapshot.export.skip.tmp
> = false) without .snapshotInfo file and manifest.
> SnapshotFileCache periodically refreshes the cache by traversing all snapshot
> directories under .snapshot and reading the .snapshotInfo file and manifest
> to update the cache. If it can not read .snapshotInfo from a snapshot
> directory, it will throw CorruptedSnapshotException.
> *getSnapshotsInProgress* may also result in such a situation when it attempts
> to retrieve the .snapshotInfo under each snapshot directory under
> .snapshot/.tmp.
> So we need to clean up the output snapshot dir before throwing exception and
> aborting this job if copying manifest fails.
> {code:java}
> cache = new SnapshotFileCache(fs, rootDir, workingFs, workingDir,
> cacheRefreshPeriod,
> cacheRefreshPeriod, "snapshot-hfile-cleaner-cache-refresher",
> new SnapshotFileCache.SnapshotFileInspector() {
> @Override
> public Collection<String> filesUnderSnapshot(final FileSystem fs,
> final Path snapshotDir)
> throws IOException {
> return SnapshotReferenceUtil.getHFileNames(conf, fs, snapshotDir);
> }
> });
> public static Set<String> getHFileNames(final Configuration conf, final
> FileSystem fs,
> final Path snapshotDir) throws IOException {
> SnapshotDescription desc = SnapshotDescriptionUtils.readSnapshotInfo(fs,
> snapshotDir);
> return getHFileNames(conf, fs, snapshotDir, desc);
> }
> public static SnapshotDescription readSnapshotInfo(FileSystem fs, Path
> snapshotDir)
> throws CorruptedSnapshotException {
> Path snapshotInfo = new Path(snapshotDir, SNAPSHOTINFO_FILE);
> try (FSDataInputStream in = fs.open(snapshotInfo)) {
> return SnapshotDescription.parseFrom(in);
> } catch (IOException e) {
> throw new CorruptedSnapshotException("Couldn't read snapshot info
> from:" + snapshotInfo, e);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
