Jackie-Jiang commented on a change in pull request #7450:
URL: https://github.com/apache/pinot/pull/7450#discussion_r713481255
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/combine/GroupByOrderByCombineOperator.java
##########
@@ -125,43 +119,41 @@ public String getOperatorName() {
*/
@Override
protected void processSegments(int threadIndex) {
- try {
- IntermediateResultsBlock intermediateResultsBlock =
- (IntermediateResultsBlock) _operators.get(threadIndex).nextBlock();
-
- _initLock.lock();
- try {
- if (_dataSchema == null) {
- _dataSchema = intermediateResultsBlock.getDataSchema();
- // NOTE: Use trimSize as resultSize on server size.
- if (_trimThreshold >= MAX_TRIM_THRESHOLD) {
- // special case of trim threshold where it is set to max value.
- // there won't be any trimming during upsert in this case.
- // thus we can avoid the overhead of read-lock and write-lock
- // in the upsert method.
- _indexedTable = new UnboundedConcurrentIndexedTable(_dataSchema,
_queryContext, _trimSize);
- } else {
- _indexedTable =
- new ConcurrentIndexedTable(_dataSchema, _queryContext,
_trimSize, _trimSize, _trimThreshold);
+ for (int operatorIndex = threadIndex; operatorIndex < _numOperators;
operatorIndex += _numThreads) {
Review comment:
The existing code in the group-by combine operator has the assumption
that one thread only processes one segment. This PR removes that assumption,
and each thread can process multiple segments. Currently we still use
`_numOperators` threads to process the query, but if we give less threads, it
still works
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
