Jackie-Jiang commented on code in PR #15431:
URL: https://github.com/apache/pinot/pull/15431#discussion_r2029551158
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java:
##########
@@ -1450,9 +1450,15 @@ public void goOnlineFromConsuming(SegmentZKMetadata
segmentZKMetadata)
.info("Current offset {} matches offset in zk {}.
Replacing segment", _currentOffset, endOffset);
buildSegmentAndReplace();
} else {
- _segmentLogger.info("Attempting to catch up from offset {} to
{} ", _currentOffset, endOffset);
- boolean success = catchupToFinalOffset(endOffset,
-
TimeUnit.MILLISECONDS.convert(MAX_TIME_FOR_CONSUMING_TO_ONLINE_IN_SECONDS,
TimeUnit.SECONDS));
+ boolean success = false;
+ // since online helix transition for a segment can arrive
before segment's consumer acquires the
+ // semaphore, check _consumerSemaphoreAcquired before catching
up.
+ // This is to avoid consuming in parallel to another segment's
consumer.
+ if (_consumerSemaphoreAcquired.get()) {
+ _segmentLogger.info("Attempting to catch up from offset {}
to {} ", _currentOffset, endOffset);
+ success = catchupToFinalOffset(endOffset,
+
TimeUnit.MILLISECONDS.convert(MAX_TIME_FOR_CONSUMING_TO_ONLINE_IN_SECONDS,
TimeUnit.SECONDS));
+ }
Review Comment:
Let's add a warning if the consumer hasn't acquired the lock for debugging
purpose
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java:
##########
@@ -1450,9 +1450,15 @@ public void goOnlineFromConsuming(SegmentZKMetadata
segmentZKMetadata)
.info("Current offset {} matches offset in zk {}.
Replacing segment", _currentOffset, endOffset);
buildSegmentAndReplace();
} else {
- _segmentLogger.info("Attempting to catch up from offset {} to
{} ", _currentOffset, endOffset);
- boolean success = catchupToFinalOffset(endOffset,
-
TimeUnit.MILLISECONDS.convert(MAX_TIME_FOR_CONSUMING_TO_ONLINE_IN_SECONDS,
TimeUnit.SECONDS));
+ boolean success = false;
+ // since online helix transition for a segment can arrive
before segment's consumer acquires the
Review Comment:
(minor) Use capital case for the first letter for comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
