Junegunn Choi created HBASE-30440:
-------------------------------------
Summary: Skip archived master-local WALs in WALInputFormat
Key: HBASE-30440
URL: https://issues.apache.org/jira/browse/HBASE-30440
Project: HBase
Issue Type: Improvement
Reporter: Junegunn Choi
h2. Problem
When WALPlayer scans {{oldWALs}}, {{WALInputFormat}} creates input splits for
archived master-local WALs ending in {{$masterlocalwal$}}. These files contain
internal {{master:store}} edits and are not needed for table replay. Actual
{{hbase:meta}} edits use separate {{.meta}} WALs.
{{WALInputFormat.addFile}} treats these filenames as having no parseable
timestamp and includes them regardless of the requested time range. Selecting a
table only filters edits in the mapper, after the input files have been opened.
In one of our test clusters, there were 683 master-local WALs among 777 regular
WAL files.
h2. Fix
Skip files ending in {{MasterRegionFactory.ARCHIVED_WAL_SUFFIX}} before
creating input splits. Preserve ordinary WALs, {{.meta}} WALs, and recovered
edits. HBASE-26300 already excludes archived master-local WALs from incremental
backups.
Regression coverage checks recursive directory and explicit-file inputs, empty
and non-empty master-local WALs, and both settings of
{{wal.input.ignore.empty.files}}. All 10 tests in {{TestWALInputFormat}} and
{{TestWALRecordReader}} pass. Spotless also passes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
