Jackie-Jiang commented on code in PR #14443:
URL: https://github.com/apache/pinot/pull/14443#discussion_r1841342429
##########
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common/SegmentGenerationTaskRunner.java:
##########
@@ -167,19 +171,25 @@ private SegmentNameGenerator
getSegmentNameGenerator(SegmentGeneratorConfig segm
return new
InputFileSegmentNameGenerator(segmentNameGeneratorConfigs.get(FILE_PATH_PATTERN),
segmentNameGeneratorConfigs.get(SEGMENT_NAME_TEMPLATE),
inputFileUri, appendUUIDToSegmentName);
case BatchConfigProperties.SegmentNameGeneratorType.UPLOADED_REALTIME:
- Preconditions.checkState(segmentGeneratorConfig.getCreationTime() !=
null,
+
Preconditions.checkState(segmentNameGeneratorConfigs.get(SEGMENT_UPLOAD_TIME_MS)
!= null,
"Creation time must be set for uploaded realtime segment name
generator");
-
Preconditions.checkState(segmentGeneratorConfig.getUploadedSegmentPartitionId()
!= -1,
+
Preconditions.checkState(segmentNameGeneratorConfigs.get(SEGMENT_PARTITION_ID)
!= null,
"Valid partition id must be set for uploaded realtime segment name
generator");
long creationTime;
Review Comment:
Suggest renaming this as it is upload time instead of creation time.
##########
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common/SegmentGenerationTaskRunner.java:
##########
@@ -62,6 +62,10 @@ public class SegmentGenerationTaskRunner implements
Serializable {
public static final String FILE_PATH_PATTERN = "file.path.pattern";
public static final String SEGMENT_NAME_TEMPLATE = "segment.name.template";
+ // For UploadedRealtimeSegmentNameGenerator
+ public static final String SEGMENT_PARTITION_ID = "segment.partitionId";
+ public static final String SEGMENT_UPLOAD_TIME_MS = "segment.uploadTimeMs";
Review Comment:
These 2 are also configured in `BatchConfigProperties`. Consider directly
use the ones there
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
