zhangshuyan0 opened a new pull request, #5622: URL: https://github.com/apache/hadoop/pull/5622
### Description of PR `processFirstBlockReport()` is used to process first block report from datanode. It does not calculating `toRemove` list because it believes that there is no metadata about the datanode in the namenode. However, If a datanode is re registered after restarting, its `blockReportCount` will be updated to 0. https://github.com/apache/hadoop/blob/c7699d3dcd4f8feaf2c5ae5943b8a4cec738e95d/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L1007-L1012 That is to say, the first block report after a datanode restarts will be processed by `processFirstBlockReport()`. https://github.com/apache/hadoop/blob/c7699d3dcd4f8feaf2c5ae5943b8a4cec738e95d/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L2916-L2925 This is unreasonable because the metadata of the datanode already exists in namenode at this time, and if redundant replica metadata is not removed in time, the blocks with insufficient replicas cannot be reconstruct in time, which increases the risk of missing block. In summary, `processFirstBlockReport()` should only be used when the namenode restarts, not when the datanode restarts. ### How was this patch tested? Add a new unit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
