[
https://issues.apache.org/jira/browse/HDFS-17777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18063944#comment-18063944
]
ASF GitHub Bot commented on HDFS-17777:
---------------------------------------
balodesecurity opened a new pull request, #8325:
URL: https://github.com/apache/hadoop/pull/8325
## Summary
`ExcessRedundancyMap` uses a single coarse-grained `synchronized` monitor
for all operations. Because `contains()`, `getSize4Testing()`, and
`getExcessRedundancyMap()` are purely read-only, they unnecessarily serialize
against each other under the block management write lock, reducing throughput
on read-heavy workloads.
### Changes
- Replace `synchronized` on every method with an explicit
`ReentrantReadWriteLock`:
- **Read lock**: `contains()`, `getSize4Testing()`,
`getExcessRedundancyMap()`
- **Write lock**: `add()`, `remove()`, `clear()`
- The `size` counter is already an `AtomicLong` and requires no additional
locking.
### Test
Added `TestExcessRedundancyMap` with 4 unit tests:
- `testAddContainsRemove` — basic correctness of add/contains/remove
- `testClear` — clear resets map and size counter
- `testConcurrentReads` — 8 concurrent readers all complete within 10 s (no
deadlock)
- `testConcurrentWritesProduceCorrectSize` — 10 concurrent writers produce a
consistent final size
All 4 tests pass.
## Test plan
- [x] `TestExcessRedundancyMap` (4 tests) — PASS
- [ ] Full `blockmanagement` test suite (CI)
> Improve ExcessRedundancyMap locking semantics
> ---------------------------------------------
>
> Key: HDFS-17777
> URL: https://issues.apache.org/jira/browse/HDFS-17777
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 3.4.0, 3.3.6, 3.4.1
> Reporter: William Montaz
> Priority: Major
> Attachments: Capture d’écran 2025-04-30 à 14.32.41.png
>
>
> ExcessRedundancyMap introduce in HDFS-9838 uses synchronized keyword for
> threadsafety. However prior to introduce this class, the operations such as
> contains were made without a new lock, they were called inside BlockManager
> with verification that the fsNamesystem lock was held in read or write mode
> depending on the situation.
>
> ExcessRedundancyMap now forces all thread to grab the same exclusive lock,
> even if in general a lot more read are performed on the class.
>
> By using a ReentrantReadWriteLock for ExcessRedundancyMap we could improve
> throughput of the namenode. Another approach could be to introduce the same
> asserts on namesystem lock as it seems those methods are always called in the
> context of the FSNamesystem lock.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
