deemoliu commented on PR #10047: URL: https://github.com/apache/pinot/pull/10047#issuecomment-1438970872
> Do we plan to implement the "Remove synchronously" approach described in the design doc? If so, it cannot be done in the replace segment because it is too late, and the new consuming segment might already be created. We will need to add a separate method which is invoked after the segment is getting committed, and before the new segment starts consuming. Hi @Jackie-Jiang I checked this part, the function will be called during LLRealtimeSegmentDataManager.commitSegment(String, boolean). Since it's triggered when commit Segment, the new segment on the same partition should not started consuming. Can you help me understand why the new consuming segment might already be created? Do you foresee any risk on the following operations during segment commit? removeExpiredPrimaryKeys(segment, expiredTimestamp); persistSnapshotForStableSegment(expiredTimestamp); -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
