siddharthteotia commented on a change in pull request #6886:
URL: https://github.com/apache/incubator-pinot/pull/6886#discussion_r630468400
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/combine/BaseCombineOperator.java
##########
@@ -127,7 +127,7 @@ public void runJob() {
// Deregister the main thread and wait for all threads done
phaser.awaitAdvance(phaser.arriveAndDeregister());
}
- CombineOperatorUtils.setExecutionStatistics(mergedBlock, _operators,
totalWorkerThreadCpuTimeNs.get());
+ CombineOperatorUtils.setExecutionStatistics(mergedBlock, _operators,
totalWorkerThreadCpuTimeNs.get(), _numThreads);
Review comment:
Let's say there are 8 tasks and 4 threads in the executor service. At
any point in time, the executor service will run 4 worker threads/tasks on CPU
assuming numCores >= 4
So, let's say following happens
task 1 - thread 1 -> threadCpuTime - t1
task 2 - thread 2 -> threadCpuTime - t2
task 3 - thread 3 -> threadCpuTime - t3
task 4 - thread 4 -> threadCpuTime - t4
task 5 - thread 1 -> threadCpuTime - t5
task 6 - thread 2 -> threadCpuTime - t6
task 7 - thread 3 -> threadCpuTime - t7
task 8 - thread 4 -> threadCpuTime - t8
Today, we compute the total as sum (t1... t8) to get total threadCpuTime
across all query threads.
So, in the adjustment equation when we do threadCpuTime / N , should N be
used as 8 or 4
I think N should be 4 because number of tasks is dependent on the number of
operators (segments)
GroupBy operator passes _numThreads as numOperators to BaseCombineOperator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
