deemoliu commented on code in PR #9062:
URL: https://github.com/apache/pinot/pull/9062#discussion_r928002601
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java:
##########
@@ -416,15 +415,18 @@ private void buildDedupMeta(ImmutableSegmentImpl
immutableSegment) {
private void handleUpsert(ImmutableSegmentImpl immutableSegment) {
String segmentName = immutableSegment.getSegmentName();
- Integer partitionGroupId =
- SegmentUtils.getRealtimeSegmentPartitionId(segmentName,
_tableNameWithType, _helixManager,
- _primaryKeyColumns.get(0));
- Preconditions.checkNotNull(partitionGroupId,
- String.format("PartitionGroupId is not available for segment: '%s'
(upsert-enabled table: %s)", segmentName,
+ Integer partitionGroupId = SegmentUtils
+ .getRealtimeSegmentPartitionId(segmentName, _tableNameWithType,
_helixManager, _primaryKeyColumns.get(0));
+ Preconditions.checkNotNull(partitionGroupId, String
+ .format("PartitionGroupId is not available for segment: '%s'
(upsert-enabled table: %s)", segmentName,
_tableNameWithType));
PartitionUpsertMetadataManager partitionUpsertMetadataManager =
_tableUpsertMetadataManager.getOrCreatePartitionManager(partitionGroupId);
+
ThreadSafeMutableRoaringBitmap validDocIds = new
ThreadSafeMutableRoaringBitmap();
+ if (immutableSegment.getValidDocIds() != null) {
Review Comment:
Gotcha, thanks for review.
For immutableSegmentImpl, it already has `validDocsIds` variable for keeping
in memory states, I was think of loading the validDocsSnapshots from disk to
`validDocsIds` variable directly. Is there any concern/risk on this?
I can also add another variable call validDocsSnapshots that loaded from
disk. Iterate over validDocsSnapshots to get the recordInfoInteractor and
re-generate validDocIds in PartitionUpsertMetadataManager (addSegment).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
