steveloughran commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1191326991
This is my draft commit message btw
----
Adds a new IOStatistics class IOStatisticsContext.
This is the active collector of thread-level statistics for
The current thread.
The S3A Filesystem's input and output streams, and listing
operations, all update this context when close() is called on
them (and not before!), so there is effectively automatic
aggregation of all IO statistics performed by a single thread.
The IOStatisticsContext of a thread can be retrieved and
cached for invocation in other threads. Holding such a
reference also ensures that the context will not be garbage
collected.
To collect statistics on a thread:
1. Retrieve the active context with a call to
IOStatisticsContext.getCurrentIOStatisticsContext()
2. Call IOStatisticsContext.reset() to reset all statistics
3. Call getIOStatistics() on it for the latest values, or
snapshot() for a snapshot of them.
To instrument filesystem objects for thread-level
IOStatistics
1. Cache the current IOStatisticsContext context or just its
aggregator in the object constructor.
2. In the close() operation, aggregate() the object's own statistics.
3. Pass the context into worker threads performing work
on behalf of this thread, through
IOStatisticsContext.setThreadIOStatisticsContext();
set it to null afterwards.
TaskPool does the context propagation and reset
automatically.
Contributed by Mehakmeet Singh
---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
