anmolanmol1234 opened a new pull request, #7669:
URL: https://github.com/apache/hadoop/pull/7669
Enhance the performance of ABFS Driver for write-heavy workloads by
improving concurrency within writes.
The proposed design advocates for a centralized `WriteThreadPoolSizeManager`
class to handle the collective thread allocation required for all write
operations across the system, replacing the current CachedThreadPool in
AzureBlobFileSystemStore. This centralized approach ensures that the initial
thread pool size is set at `4 * number of available processors` and dynamically
adjusts the pool size based on the system's current CPU utilization. This
adaptive scaling and descaling mechanism optimizes resource usage and
responsiveness. Moreover, this shared thread pool is accessible and utilized by
all output streams, streamlining resource management and promoting efficient
concurrency across write operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
