[
https://issues.apache.org/jira/browse/HADOOP-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392860#comment-15392860
]
Mingliang Liu edited comment on HADOOP-13032 at 7/25/16 11:43 PM:
------------------------------------------------------------------
The v0 patch is the first effort of refactoring the {{FileSystem$Statistics}}
class to use the newly added {{StorageStatistics}}. Specially, it:
* Defines a new class {{ThreadLocalFsStorageStatistics}} as the thread-local
implementation to aggregate stats. This will be also used in [HADOOP-13031],
which is to separate the distance-specific rack-aware read bytes logic from
{{FileSystemStorageStatistics}} to new {{DFSRackAwareStorageStatistics}} as
it's DFS-specific.
* Makes {{FileSystemStorageStatistics}} use the new
{{ThreadLocalFsStorageStatistics}} class instead of delegating all operations
to {{FileSystem$Statistics}}
* Removes the deprecated {{FileSystem$Statistics}} class
* Updates all the usages of the {{FileSystem$Statistics}} to use
{{FileSystemStorageStatistics}}
TODO:
# This patch is large. We should split the patch into small ones. I plan to
firstly separate the first two items above out of a new JIRA, along with new
unit tests
# The MapReduce part that explores the {{FileSystemStorageStatistics}} should
also consider the rack-aware read bytes. This should be borrowing code from
[MAPREDUCE-6660]
was (Author: liuml07):
The v0 patch is the first effort of refactoring the {{FileSystem$Statistics}}
class to use the newly added {{StorageStatistics}}. Specially, it:
* Defines a new class {{ThreadLocalFsStorageStatistics}} as the thread-local
implementation to aggregate stats. This will be used in [HADOOP-13031]
* Makes {{FileSystemStorageStatistics}} use the new
{{ThreadLocalFsStorageStatistics}} class instead of delegating all operations
to {{FileSystem$Statistics}}
* Removes the deprecated {{FileSystem$Statistics}} class
* Updates all the usages of the {{FileSystem$Statistics}} to use
{{FileSystemStorageStatistics}}
TODO:
# This patch is large. We should split the patch into small ones. I plan to
firstly separate the first two items above out of a new JIRA, along with new
unit tests
# The MapReduce part that explores the {{FileSystemStorageStatistics}} should
also consider the rack-aware read bytes. This should be borrowing code from
[MAPREDUCE-6660]
> Refactor FileSystem$Statistics to use StorageStatistics
> -------------------------------------------------------
>
> Key: HADOOP-13032
> URL: https://issues.apache.org/jira/browse/HADOOP-13032
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Attachments: HADOOP-13032.000.patch
>
>
> [HADOOP-13065] added a new interface for retrieving FS and FC Statistics.
> This jira is to track the effort of moving the {{Statistics}} class out of
> {{FileSystem}}, and make it use that new interface.
> We should keep the thread local implementation. Benefits are:
> # they could be used in both {{FileContext}} and {{FileSystem}}
> # unified stats data structure
> # shorter source code
> Please note this will be an backwards-incompatible change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
