klsince commented on code in PR #13992:
URL: https://github.com/apache/pinot/pull/13992#discussion_r1764087728
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java:
##########
@@ -1200,12 +1219,45 @@ public void trackSegmentForUpsertView(IndexSegment
segment) {
if (_upsertViewManager != null) {
_upsertViewManager.trackSegment(segment);
}
+ if (segment instanceof MutableSegment) {
+ trackNewlyAddedSegment(segment);
+ }
}
@Override
public void untrackSegmentForUpsertView(IndexSegment segment) {
if (_upsertViewManager != null) {
_upsertViewManager.untrackSegment(segment);
}
+ if (segment instanceof MutableSegment) {
+ untrackNewlyAddedSegment(segment);
+ }
+ }
+
+ @VisibleForTesting
+ void trackNewlyAddedSegment(IndexSegment segment) {
+ if (_newSegmentTrackingTimeMs > 0) {
+ _newlyAddedSegments.put(segment.getSegmentName(), -1L);
+ }
+ }
+
+ @VisibleForTesting
+ void untrackNewlyAddedSegment(IndexSegment segment) {
Review Comment:
As just tested, broker reacts on IS changes, and it tracks newly uploaded
segments as 'New' and add them to the list of optional segments for server to
process. The newly uploaded segments are added to broker's routing table once
IS gets updated, and it doesn't wait for the segment to get ONLINE in EV.
you're right here
> IMO that might not be necessary because we use this timeout to give some
buffer time to broker to reflect the IS change....
I can simplify the logic in this PR to kick off the tracking timer when
starting to add the segment, instead of waiting for adding segment to finish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
