vrajat opened a new issue, #16042:
URL: https://github.com/apache/pinot/issues/16042
For every query execution thread, a ThreadResourceUsageProvider is created.
Example for a worker:
```
public static void setupWorker(int taskId,
ThreadExecutionContext.TaskType taskType,
@Nullable ThreadExecutionContext threadExecutionContext) {
Tracing.getThreadAccountant().setThreadResourceUsageProvider(new
ThreadResourceUsageProvider());
Tracing.getThreadAccountant().setupWorker(taskId, taskType,
threadExecutionContext);
}
```
The ThreadResourceUsageAccountant stores it in a thread local.
```
// ThreadResourceUsageProvider(ThreadMXBean wrapper) per runner/worker
thread
private final ThreadLocal<ThreadResourceUsageProvider>
_threadResourceUsageProvider;
```
However take a look at the code to get the memory allocation for a thread.
```
public long getThreadAllocatedBytes() {
try {
return _isThreadMemoryMeasurementEnabled ? (long)
SUN_THREAD_MXBEAN_GET_BYTES_ALLOCATED_METHOD
.invoke(MX_BEAN, Thread.currentThread().getId()) -
_startBytesAllocated : 0;
} catch (IllegalAccessException | InvocationTargetException e) {
LOGGER.error("Exception happened during the invocation of getting
initial bytes allocated", e);
return 0;
}
}
```
It is passing the id of the current thread: Thread.currentThread().getId() .
So the function itself is not using any thread local information.
There are a couple of non-static member variables in
`ThreadResourceUsageProvider`.
```
// reference point for start time/bytes
private final long _startTimeNs;
private final long _startBytesAllocated;
```
These can be easily moved to
`CPUMemThreadLevelAccountingObjects.ThreadEntry` class
Note that a large number of NPE was because `_threadResourceUsageProvider`
was not set. So this change will fix a class of issues by getting rid of this
per-thread object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
