deemoliu commented on code in PR #10047:
URL: https://github.com/apache/pinot/pull/10047#discussion_r1198137711
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java:
##########
@@ -526,6 +529,32 @@ private void handleUpsert(ImmutableSegment
immutableSegment) {
_tableNameWithType));
PartitionUpsertMetadataManager partitionUpsertMetadataManager =
_tableUpsertMetadataManager.getOrCreatePartitionManager(partitionId);
+ return partitionUpsertMetadataManager;
+ }
+
+ /**
+ * Handle Upsert TTL during segment state change from CONSUMING to ONLINE.
+ *
+ * When a new segments commit, we will remove primary key indexes from heap
that has timestamp < (eventTime - TTL).
+ * For all segments sealed before (eventTime-TTL), their validDocIds won't
get updated. We do an one-time persistance.
Review Comment:
Only segments that haven't been persisted before will do snapshot
persistence.
We have a concurrent priorityQueue to keep track all the segments that
snapshot not have been persisted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
