KKcorps commented on code in PR #9825:
URL: https://github.com/apache/pinot/pull/9825#discussion_r1036044210
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/MinionTaskUtils.java:
##########
@@ -67,7 +72,35 @@ static PinotFS getOutputPinotFS(Map<String, String>
taskConfigs, URI fileURI)
return PinotFSFactory.create(fileURIScheme);
}
- static PinotFS getLocalPinotFs() {
+ public static Map<String, String> getPushTaskConfig(String tableName,
Map<String, String> taskConfigs,
+ ClusterInfoAccessor clusterInfoAccessor) {
+ try {
+ URI outputDirURI = URI.create(clusterInfoAccessor.getDataDir() + "/" +
tableName);
Review Comment:
It is not really used in the case when pushMode is TAR.
```java
if (BatchConfigProperties.SegmentPushType.valueOf(pushMode.toUpperCase())
!= BatchConfigProperties.SegmentPushType.TAR) {
outputSegmentTarURI = moveSegmentToOutputPinotFS(configs,
convertedTarredSegmentFile);
LOGGER.info("Moved generated segment from [{}] to location: [{}]",
convertedTarredSegmentFile,
outputSegmentTarURI);
} else {
outputSegmentTarURI = convertedTarredSegmentFile.toURI();
}
......
case TAR:
try (PinotFS pinotFS = MinionTaskUtils.getLocalPinotFs()) {
SegmentPushUtils.pushSegments(
spec, pinotFS, Arrays.asList(outputSegmentTarURI.toString()),
headers, parameters);
} catch (RetriableOperationException | AttemptsExceededException e) {
throw new RuntimeException(e);
}
break;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
