Dieter De Paepe created HBASE-30542:
---------------------------------------
Summary: Recreating a table can corrupt incremental backups
Key: HBASE-30542
URL: https://issues.apache.org/jira/browse/HBASE-30542
Project: HBase
Issue Type: Bug
Components: backup&restore
Reporter: Dieter De Paepe
**Summary:** Incremental backup after a table is deleted and recreated restores
data of the deleted table
```
h3. Problem
The backup system does not notice when a table is deleted and recreated with
the same name. The next incremental backup treats the new table as a
continuation of the old one, so restoring that backup image restores data of
the deleted table.
Found while working on HBASE-29905, by code inspection. I have not reproduced
this with a test yet.
h3. Scenario
# A full backup F1 includes table T in backup root R. This adds T to the
incremental backup set of R (\{{incrbackupset:}} row) and stores the WAL
timestamps of T in its \{{trslm:}} row.
# Rows are written to T.
# T is deleted. \{{BackupMasterObserver#postDeleteTable}} only removes bulk
load rows, so T stays in the incremental backup set of R and its \{{trslm:}}
row stays as well.
# T is recreated with the same name and the same column families, and new rows
are written to it.
# An incremental backup I1 runs on R. Incremental backups cover the whole
incremental backup set, and T exists again, so I1 includes T. The column family
check in \{{IncrementalTableBackupClient}} only compares column family names,
so it passes.
# T is restored from I1.
h3. Expected
Either the restored T only contains the rows of the recreated table, or I1
refuses to back up T until a new full backup of T is taken.
h3. Actual
Restoring I1 starts from the snapshot of the deleted table in F1 and then
applies the incremental data. Deleting a table produces no delete markers, so
the restored T contains:
* all rows of the deleted table from F1,
* the edits made to the deleted table between F1 and its deletion, which I1
reads from the WALs because the \{{trslm:}} timestamps of T still point to F1,
* the rows of the recreated table.
If the recreated table has different column families, I1 fails with
\{{ColumnFamilyMismatchException}} instead, so that variant does not corrupt
data.
{\{postTruncateTable}} has the same gap: truncating also removes data without
delete markers. I have not verified the truncate case.
h3. Possible fix
Treat a deleted table as no longer backed up. In
\{{BackupMasterObserver#postDeleteTable}}, remove the table from the
incremental backup set of every backup root and delete its \{{trslm:}} rows.
The next incremental backup then skips the recreated table, and a request that
names it explicitly fails with "haven't gone through full backup". The existing
backups of the deleted table stay restorable: restore reads the backup images
and manifests, and point-in-time restore reads the backup history and the
continuous backup table set, so neither uses these rows. As a side effect, this
also removes the stale \{{trslm:}} rows that HBASE-29905 works around in
BackupLogCleaner.
```
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)