jackjlli commented on a change in pull request #8110:
URL: https://github.com/apache/pinot/pull/8110#discussion_r798939093
##########
File path:
pinot-common/src/main/java/org/apache/pinot/common/metadata/ZKMetadataProvider.java
##########
@@ -141,6 +141,26 @@ public static void
removeResourceConfigFromPropertyStore(ZkHelixPropertyStore<ZN
}
}
+ /**
+ * Creates a new znode for SegmentZkMetadata. This call is atomic. If there
are concurrent calls trying to create the
+ * same znode, only one of them would succeed.
+ *
+ * @param propertyStore Helix property store
+ * @param tableNameWithType Table name with type
+ * @param segmentZKMetadata Segment Zk metadata
+ * @return boolean indicating success/failure
+ */
+ public static boolean createSegmentZkMetadata(ZkHelixPropertyStore<ZNRecord>
propertyStore, String tableNameWithType,
+ SegmentZKMetadata segmentZKMetadata) {
+ try {
+ return propertyStore
+ .create(constructPropertyStorePathForSegment(tableNameWithType,
segmentZKMetadata.getSegmentName()),
+ segmentZKMetadata.toZNRecord(), AccessOption.PERSISTENT);
+ } catch (Exception e) {
+ return false;
Review comment:
Could we add a message here when any exception is thrown from the ZK?
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -1778,48 +1778,69 @@ public void addNewSegment(String tableNameWithType,
SegmentMetadata segmentMetad
public void addNewSegment(String tableNameWithType, SegmentMetadata
segmentMetadata, String downloadUrl,
@Nullable String crypter) {
- String segmentName = segmentMetadata.getName();
- InstancePartitionsType instancePartitionsType;
// NOTE: must first set the segment ZK metadata before assigning segment
to instances because segment assignment
// might need them to determine the partition of the segment, and server
will need them to download the segment
- ZNRecord znRecord;
+ SegmentZKMetadata segmentZkmetadata =
+ constructZkMetadataForNewSegment(tableNameWithType, segmentMetadata,
downloadUrl, crypter);
+ ZNRecord znRecord = segmentZkmetadata.toZNRecord();
+
+ String segmentName = segmentMetadata.getName();
+ String segmentZKMetadataPath =
+
ZKMetadataProvider.constructPropertyStorePathForSegment(tableNameWithType,
segmentName);
+ Preconditions.checkState(_propertyStore.set(segmentZKMetadataPath,
znRecord, AccessOption.PERSISTENT),
+ "Failed to set segment ZK metadata for table: " + tableNameWithType +
", segment: " + segmentName);
+ LOGGER.info("Added segment: {} of table: {} to property store",
segmentName, tableNameWithType);
+
+ assignTableSegment(tableNameWithType, segmentName);
+ }
+
+ /**
+ * Construct segmentZkMetadata for the realtime or offline table.
Review comment:
for new segment of realtime or offline table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
