npawar commented on a change in pull request #7294:
URL: https://github.com/apache/pinot/pull/7294#discussion_r688193207
##########
File path:
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentLoader.java
##########
@@ -169,7 +158,20 @@ public static ImmutableSegment load(File indexDir,
IndexLoadingConfig indexLoadi
ImmutableSegmentImpl segment =
new ImmutableSegmentImpl(actualSegmentDirectory, segmentMetadata,
indexContainerMap, starTreeIndexContainer);
- LOGGER.info("Successfully loaded segment {} with config: {}", segmentName,
segmentDirectoryLoaderConfigs);
+ LOGGER.info("Successfully loaded segment {} with config: {}", segmentName,
segDirConfigs);
return segment;
}
+
+ // Pre-process the segment on local using local SegmentDirectory.
+ // Please note that this step may modify the segment metadata.
+ private static void preprocessSegment(File indexDir, IndexLoadingConfig
indexLoadingConfig, Schema schema)
+ throws Exception {
+ PinotConfiguration tierConfigs = indexLoadingConfig.getTierConfigs();
+ PinotConfiguration segDirConfigs = new
PinotConfiguration(tierConfigs.toMap());
+ SegmentDirectory segDir =
+
SegmentDirectoryLoaderRegistry.getLocalSegmentDirectoryLoader().load(indexDir.toURI(),
segDirConfigs);
Review comment:
you're right in terms of what the LocalSegmentDirectory needs. I choose
to make the `getTierConfigs` be the call that returns all tier related
properties. `loadMode` is a tier config for `local` tier. This way I could keep
the `getLocalSegmentDirectory` call's arguments consistent across the tiers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
