npawar commented on a change in pull request #7294:
URL: https://github.com/apache/pinot/pull/7294#discussion_r692373519
##########
File path:
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentLoader.java
##########
@@ -169,7 +158,20 @@ public static ImmutableSegment load(File indexDir,
IndexLoadingConfig indexLoadi
ImmutableSegmentImpl segment =
new ImmutableSegmentImpl(actualSegmentDirectory, segmentMetadata,
indexContainerMap, starTreeIndexContainer);
- LOGGER.info("Successfully loaded segment {} with config: {}", segmentName,
segmentDirectoryLoaderConfigs);
+ LOGGER.info("Successfully loaded segment {} with config: {}", segmentName,
segDirConfigs);
return segment;
}
+
+ // Pre-process the segment on local using local SegmentDirectory.
+ // Please note that this step may modify the segment metadata.
+ private static void preprocessSegment(File indexDir, IndexLoadingConfig
indexLoadingConfig, Schema schema)
+ throws Exception {
+ PinotConfiguration tierConfigs = indexLoadingConfig.getTierConfigs();
+ PinotConfiguration segDirConfigs = new
PinotConfiguration(tierConfigs.toMap());
+ SegmentDirectory segDir =
+
SegmentDirectoryLoaderRegistry.getLocalSegmentDirectoryLoader().load(indexDir.toURI(),
segDirConfigs);
Review comment:
I think i wanted to make it clear that segmentDirectoryConfigs are made
using tierConfigs, and they are not directly equivalent. But they are same for
practical purposes of this PR. Feel free to change it if you wish
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
