pengding-stripe commented on code in PR #14443:
URL: https://github.com/apache/pinot/pull/14443#discussion_r1842592785
##########
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common/SegmentGenerationTaskRunner.java:
##########
@@ -62,6 +62,10 @@ public class SegmentGenerationTaskRunner implements
Serializable {
public static final String FILE_PATH_PATTERN = "file.path.pattern";
public static final String SEGMENT_NAME_TEMPLATE = "segment.name.template";
+ // For UploadedRealtimeSegmentNameGenerator
+ public static final String SEGMENT_PARTITION_ID = "segment.partitionId";
+ public static final String SEGMENT_UPLOAD_TIME_MS = "segment.uploadTimeMs";
Review Comment:
To use `SegmentGeneratorConfig.segmentCreationTime` you have to call
`SegmentGeneratorConfig.setCreationTime()` here first. Are you suggesting
calling it here with system time instead of letting user customize it? I don't
think we can use `SegmentGeneratorConfig.segmentCreationTime` directly without
setting it.
Spark is quite similar, it can also run in batch mode and one run's segments
will have same timestamp in segment name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
