hfutatzhanghb opened a new pull request, #5262:
URL: https://github.com/apache/hadoop/pull/5262
We found lots of INFO level log like below:
>2022-12-30 15:31:04,169 INFO org.apache.hadoop.hdfs.StateChange: DIR*
completeFile: / is closed by
DFSClient_attempt_1671783180362_213003_m_000077_0_1102875551_1
2022-12-30 15:31:04,186 INFO org.apache.hadoop.hdfs.StateChange: DIR*
completeFile: / is closed by DFSClient_NONMAPREDUCE_1198313144_27480
It lost the real path of completeFile. Actually this is caused by :
>org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient#invokeSingle(java.lang.String,
org.apache.hadoop.hdfs.server.federation.router.RemoteMethod)
In this method, it instantiates a RemoteLocationContext object:
>RemoteLocationContext loc = new RemoteLocation(nsId, "/", "/");
and then execute: Object[] params = method.getParams(loc);
The problem is right here, becasuse we always use new RemoteParam(), so,
context.getDest() always return "/"; That's why we saw lots of incorrect
logs.
After diving into invokeSingleXXX source code, I found the following RPCs
classified as need actual src and not need actual src.
**need src path RPC:**
addBlock、abandonBlock、getAdditionalDatanode、complete
**not need src path RPC:**
updateBlockForPipeline、reportBadBlocks、getBlocks、updatePipeline、invokeAtAvailableNs（invoked
by:
getServerDefaults、getBlockKeys、getTransactionID、getMostRecentCheckpointTxId、versionRequest、getStoragePolicies）
After changes, the src can be pass to NN correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
