DieterDP-ng opened a new pull request, #8784: URL: https://github.com/apache/hbase/pull/8784
The backup code keys per-server state by host:port. WAL file names contain the lowercased hostname, while ServerName#getAddress keeps the hostname's original case. For a region server whose hostname contains a capital letter, the backup code therefore tracked the server under two keys, and lookups between them missed. Incremental backups then fell back to the oldest boundary of any server, so they could include WALs that an earlier backup already covered. Replaying those WALs could bring back deleted data, or fail the restore when the WALs held edits for a column family that was dropped since. Boundaries were also stored under the wrong key for later backups. The log cleaner missed the host's boundary and kept WALs longer than needed. Various tests were broken when running on a host with a capital letter. BackupUtils#toHostKey now lowercases the key. The WAL path parsers, the log roll (both variants) and the live-server check of the full backup use it. Reads from the backup system table and BackupBoundaries also normalize, because existing tables can hold mixed-case keys. On a collision, the newest roll result and the oldest boundary are kept. Mixed-case keys that are already stored are phased out as follows: - The per-table boundaries (trslm rows) are rewritten with lowercase keys by the next backup that includes the table. A table that is not backed up again keeps its old row. Backups with continuous backup enabled do not rewrite this row. - The roll results (rslogts rows) are never phased out, because nothing deletes these rows. Once the host rolls its WALs on the new code, its lowercase row holds the larger value, and the read keeps that value. - The boundaries copied into each backup's BackupInfo keep their mixed-case keys until that backup is deleted. BackupBoundaries normalizes them. The legacy log roll procedure looked up the previous roll result by hostname without port, so that lookup never matched. It now uses the full key. TestBackupOfflineRS now runs with a mixed-case region server hostname, so it covers this on hosts with lowercase names too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
