[
https://issues.apache.org/jira/browse/HADOOP-19472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17949139#comment-17949139
]
ASF GitHub Bot commented on HADOOP-19472:
-----------------------------------------
anmolanmol1234 opened a new pull request, #7669:
URL: https://github.com/apache/hadoop/pull/7669
Enhance the performance of ABFS Driver for write-heavy workloads by
improving concurrency within writes.
The proposed design advocates for a centralized `WriteThreadPoolSizeManager`
class to handle the collective thread allocation required for all write
operations across the system, replacing the current CachedThreadPool in
AzureBlobFileSystemStore. This centralized approach ensures that the initial
thread pool size is set at `4 * number of available processors` and dynamically
adjusts the pool size based on the system's current CPU utilization. This
adaptive scaling and descaling mechanism optimizes resource usage and
responsiveness. Moreover, this shared thread pool is accessible and utilized by
all output streams, streamlining resource management and promoting efficient
concurrency across write operations.
> ABFS: Enhance performance of ABFS driver for write-heavy workloads
> ------------------------------------------------------------------
>
> Key: HADOOP-19472
> URL: https://issues.apache.org/jira/browse/HADOOP-19472
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.1
> Reporter: Anmol Asrani
> Assignee: Anmol Asrani
> Priority: Minor
> Fix For: 3.4.1
>
>
> The goal of this work item is to enhance the performance of ABFS Driver for
> write-heavy workloads by improving concurrency within writes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
