Apache9 commented on code in PR #7617:
URL: https://github.com/apache/hbase/pull/7617#discussion_r2943848541
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java:
##########
@@ -90,6 +97,10 @@ public ReplicationSourceShipper(Configuration conf, String
walGroupId, Replicati
this.conf.getInt("replication.source.getEntries.timeout",
DEFAULT_TIMEOUT);
this.shipEditsTimeout =
this.conf.getInt(HConstants.REPLICATION_SOURCE_SHIPEDITS_TIMEOUT,
HConstants.REPLICATION_SOURCE_SHIPEDITS_TIMEOUT_DFAULT);
+ this.offsetUpdateIntervalMs =
+ conf.getLong("hbase.replication.shipper.offset.update.interval.ms",
Long.MAX_VALUE);
+ this.offsetUpdateSizeThresholdBytes =
+ conf.getLong("hbase.replication.shipper.offset.update.size.threshold",
-1L);
Review Comment:
HConstants is IA.Public so generally we do not want to add new fields to it
unless we can confirm they should be accessed by user code directly. But I
agree that we should extract these constants as a static final field.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java:
##########
@@ -215,6 +255,13 @@ private void shipEdits(WALEntryBatch entryBatch) {
entryBatch.getNbOperations(), (endTimeNs - startTimeNs) / 1000000);
}
break;
+ } catch (IOException ioe) {
+ // Offset-Persist failure is treated as fatal to this worker since it
might come from
Review Comment:
Better introduce a abortAndRestart method in ReplicationSourceShipper to
call a method in ReplicationSource to replace the current shipper with a new
one. This can avoid introducing a new monitor thread. And also, we can deal
with InterruptedException with the same way, as usually if we want to shutdown,
we will first change a running or close flag and then interrupt, so when we
want to reschedule, we can check the flag to determine whether we should quit
now.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java:
##########
@@ -148,6 +151,12 @@ public class ReplicationSource implements
ReplicationSourceInterface {
public static final int DEFAULT_WAIT_ON_ENDPOINT_SECONDS = 30;
private int waitOnEndpointSeconds = -1;
+ public static final String SHIPPER_MONITOR_INTERVAL =
Review Comment:
Since the code in shipper is all implemented by us, we can just reschedule a
new shipper when it is dead? I mean, we do not need a monitor thread to do this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
