This is an automated email from the ASF dual-hosted git repository.
mcvsubbu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new bacaed2 Guard against multiple consuming segments for same partition
(#6483)
bacaed2 is described below
commit bacaed22b4aa3eef38f951fdb098396706df444e
Author: Subbu Subramaniam <mcvsu...@users.noreply.github.com>
AuthorDate: Tue Jan 26 09:00:54 2021 -0800
Guard against multiple consuming segments for same partition (#6483)
* Guard against multiple consuming segments for same partition
Fixes being considered for issues #5559 and #5263 may end up
resurfacing some race conditions that we have strived to avoid
via automated mechanisms. The race conditions may lead to multiple
consuming segments in the same partition.
This commit is a catch-all safeguard in case some automaticl operation
interferes with other manual primitives provided, so that the table
never reaches such a state.
* Addressed review comments
---
.../core/realtime/PinotLLCRealtimeSegmentManager.java | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git
a/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
b/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
index 00433b3..42d50d0 100644
---
a/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
+++
b/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java
@@ -793,6 +793,25 @@ public class PinotLLCRealtimeSegmentManager {
LOGGER.info("Updating segment: {} to ONLINE state",
committingSegmentName);
}
+ // There used to be a race condition in pinot (caused by heavy GC on the
controller during segment commit)
+ // that ended up creating multiple consuming segments for the same stream
partition, named somewhat like
+ // tableName__1__25__20210920T190005Z and
tableName__1__25__20210920T190007Z. It was fixed by checking the
+ // Zookeeper Stat object before updating the segment metadata.
+ // These conditions can happen again due to manual operations considered
as fixes in Issues #5559 and #5263
+ // The following check prevents the table from going into such a state
(but does not prevent the root cause
+ // of attempting such a zk update).
+ LLCSegmentName newLLCSegmentName = new LLCSegmentName(newSegmentName);
+ int partitionId = newLLCSegmentName.getPartitionId();
+ int seqNum = newLLCSegmentName.getSequenceNumber();
+ for (String segmentNameStr : instanceStatesMap.keySet()) {
+ LLCSegmentName llcSegmentName = new LLCSegmentName(segmentNameStr);
+ if (llcSegmentName.getPartitionId() == partitionId &&
llcSegmentName.getSequenceNumber() == seqNum) {
+ String errorMsg =
+ String.format("Segment %s is a duplicate of existing segment %s",
newSegmentName, segmentNameStr);
+ LOGGER.error(errorMsg);
+ throw new HelixHelper.PermanentUpdaterException(errorMsg);
+ }
+ }
// Assign instances to the new segment and add instances as state CONSUMING
List<String> instancesAssigned =
segmentAssignment.assignSegment(newSegmentName, instanceStatesMap,
instancePartitionsMap);
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
