[
https://issues.apache.org/jira/browse/HDFS-17474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849863#comment-17849863
]
Felix N commented on HDFS-17474:
--------------------------------
Hi [~zhanghaobo] , more detailed descriptions will be in the PR once I'm done
cleaning up the code (the code is roughly done for now but stills need some
tidying up) but here's the rough idea: the underlying data structure used in
INodeMap (and BlocksMap also) is a GSet, and it should be made thread-safe
(i.e. ThreadSafeGSet). The most simple option is a ConcurrentHashMap, but a
HashMap occupies way more memory than the current LightWeightGSet and that
makes it not viable as a solution. I have 3 options in mind:
* Just a LightWeightGSet but synchronize all its operations
* LightWeightGSet with a lock for each element, and operations on an element
can synchronize on the lock assigned to that element
* LightWeightGSet with a lock for a group of elements (grouped in powers of
2), and operations on an element can synchronize on the lock assigned to that
element. Or use a ReentrantReadWriteLock instead of synchronizing on a lock
object.
I'm still benchmarking but the implementation I'm leaning towards right now is
the 3rd option with lock objects.
> [FGL] Make INodeMap thread safe
> -------------------------------
>
> Key: HDFS-17474
> URL: https://issues.apache.org/jira/browse/HDFS-17474
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: ZanderXu
> Assignee: Felix N
> Priority: Major
>
> Operations related to INodeMap should be handled by namenode safety, since
> operations maybe access or update INodeMap concurrently.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
