Jackie-Jiang commented on PR #10047: URL: https://github.com/apache/pinot/pull/10047#issuecomment-1440672679
> Hi @Jackie-Jiang I checked this part, the function will be called during LLRealtimeSegmentDataManager.commitSegment(String, boolean). Since it's triggered when commit Segment, the new segment on the same partition should not started consuming. Can you help me understand why the new consuming segment might already be created? > > Do you foresee any risk on the following operations during segment commit? > > removeExpiredPrimaryKeys(segment, expiredTimestamp); persistSnapshotForStableSegment(expiredTimestamp); Only the committing server will call `commitSegment()`, but we need all servers to clean up the expired keys after finishing the current consuming segment and before starting the new consuming segment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
