[
https://issues.apache.org/jira/browse/HDFS-17458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837206#comment-17837206
]
ASF GitHub Bot commented on HDFS-17458:
---------------------------------------
zhangshuyan0 commented on code in PR #6717:
URL: https://github.com/apache/hadoop/pull/6717#discussion_r1565660296
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java:
##########
@@ -120,15 +118,13 @@ ReplicaInfo get(String bpid, long blockId) {
ReplicaInfo add(String bpid, ReplicaInfo replicaInfo) {
checkBlockPool(bpid);
checkBlock(replicaInfo);
- try (AutoCloseDataSetLock l = lockManager.readLock(LockLevel.BLOCK_POOl,
bpid)) {
- LightWeightResizableGSet<Block, ReplicaInfo> m = map.get(bpid);
- if (m == null) {
- // Add an entry for block pool if it does not exist already
- map.putIfAbsent(bpid, new LightWeightResizableGSet<Block,
ReplicaInfo>());
- m = map.get(bpid);
- }
- return m.put(replicaInfo);
+ LightWeightResizableGSet<Block, ReplicaInfo> m = map.get(bpid);
+ if (m == null) {
+ // Add an entry for block pool if it does not exist already
+ map.putIfAbsent(bpid, new LightWeightResizableGSet<Block,
ReplicaInfo>());
+ m = map.get(bpid);
}
+ return m.put(replicaInfo);
Review Comment:
It's not safe here. If there is somebody changing the `map` after line125
but before line127, the `replicaInfo` may not be able to added to `map`.
> Remove unnecessary BP lock in ReplicaMap
> ----------------------------------------
>
> Key: HDFS-17458
> URL: https://issues.apache.org/jira/browse/HDFS-17458
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.4.0
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Major
> Labels: pull-request-available
>
> In HDFS-16429 we make LightWeightResizableGSet to be thread safe, and in
> HDFS-16511 we change some methods in ReplicaMap to acquire read lock instead
> of acquiring write lock.
> This PR try to remove unnecessary Block_Pool read lock further.
> Recently, I performed stress tests on datanodes to measure their read/write
> operations/second.
> Before we removing some lock, it can only achieve ~2K write ops. After
> optimizing, it can achieve more than 5K write ops.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
