[
https://issues.apache.org/jira/browse/HBASE-29611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18028100#comment-18028100
]
Anuj Sharma commented on HBASE-29611:
-------------------------------------
[~wchevreuil] your below view point is corrrect, and this is what happening as
of now. So to prevent that we have added this fix.
{quote}In any case, replica cluster must never update or create any file list
file ever, as this could lead to data loss.
{quote}
Cause:
The refresh store file is a two step process:
# Load the existing store file from the .filelist (choose the file with higher
timestamp for loading)
# refresh store file internals (clean up old/compacted files, replace store
file in .filelist)
In the current scenario, what is happening is that for the first time
read-replica is loading the list of Hfiles from the file in .filelist created
by active cluster but then it is creating the new file with greater timestamp.
Now we have two files in .filelist. On the subsequent flush from active the
file in .filelist created by the active gets updated but the file created by
read-replica is not. While loading again (from read replica) in the
refresh_hfiles as we take the file with higher timestamp, which is the file
created by read-replica for the first time, gets loaded which does not have an
updated list of hfiles.
Fix:
As we just wanted the file from active to be loaded anytime we perform refresh
store files, we must not create a new file in the .filelist from the
read-replica, in this way we will stop the timestamp mismatch.
Also we don't want to initialize the tracker file
(StoreFileListFile.java:load()) from read-replica as we are not writing it
hence we have added check for read only property in
StoreFileTrackerBase.java:load()
> With FILE based SFT, the list of HFiles we maintain in .filelist is not
> getting updated for read replica
> --------------------------------------------------------------------------------------------------------
>
> Key: HBASE-29611
> URL: https://issues.apache.org/jira/browse/HBASE-29611
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anuj Sharma
> Assignee: Anuj Sharma
> Priority: Major
>
> Repro steps:
> * Create two clusters on same storage location
> * Created table on active cluster
> * Refreshed meta on replica -> can list the new table
> * Added some rows, flushed the table on active cluster
> * Refreshed hfiles on replica -> can see the data rows
> * Added one more row and flushed the table again. Another hfile was created.
>
> {noformat}
> drwxr-xr-x 3 andor staff 96 Sep 12 11:43 .filelist
> -rw-r--r-- 1 andor staff 5024 Sep 12 11:41 2539b1fd589b43d392b333896a615911
> -rw-r--r-- 1 andor staff 4956 Sep 12 11:42
> ea70ac19c6bc42bcb1ac33961b1605b3{noformat}
>
> * Refresh hfiles again on replica cluster, but can't see the new row. Also
> if you check .filelist it does not have newly added hfiles entry.
> {noformat}
> $ cat
> hbase/data/default/andor/97f8ce9ed533d7508ecc88876601aa60/cf/.filelist/f2.1757695318604
> 鴸3%
> 2539b1fd589b43d392b333896a615911'%
> 2539b1fd589b43d392b333896a615911'{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
