yashmayya commented on code in PR #17057:
URL: https://github.com/apache/pinot/pull/17057#discussion_r2456462366
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -120,11 +123,18 @@ public class SegmentIndexCreationDriverImpl implements
SegmentIndexCreationDrive
private int _incompleteRowsFound = 0;
private int _skippedRowsFound = 0;
private int _sanitizedRowsFound = 0;
+ private @Nullable InstanceType _instanceType;
@Override
public void init(SegmentGeneratorConfig config)
throws Exception {
- init(config, getRecordReader(config));
+ init(config, getRecordReader(config), null);
Review Comment:
If this one is just for tests let's annotate it with `@VisibleForTesting`? I
don't think this init method should be called from any non test / benchmark
code paths right?
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -120,11 +123,18 @@ public class SegmentIndexCreationDriverImpl implements
SegmentIndexCreationDrive
private int _incompleteRowsFound = 0;
private int _skippedRowsFound = 0;
private int _sanitizedRowsFound = 0;
+ private @Nullable InstanceType _instanceType;
Review Comment:
```suggestion
@Nullable private InstanceType _instanceType;
```
nit: annotation convention
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -168,16 +178,23 @@ public RecordReader getRecordReader() {
public void init(SegmentGeneratorConfig config, RecordReader recordReader)
throws Exception {
init(config, new RecordReaderSegmentCreationDataSource(recordReader),
- new TransformPipeline(config.getTableConfig(), config.getSchema()));
+ new TransformPipeline(config.getTableConfig(), config.getSchema()),
null);
+ }
+
+ public void init(SegmentGeneratorConfig config, RecordReader recordReader,
@Nullable InstanceType instanceType)
+ throws Exception {
+ init(config, new RecordReaderSegmentCreationDataSource(recordReader),
+ new TransformPipeline(config.getTableConfig(), config.getSchema()),
instanceType);
}
public void init(SegmentGeneratorConfig config, SegmentCreationDataSource
dataSource,
- TransformPipeline transformPipeline)
+ TransformPipeline transformPipeline, @Nullable InstanceType instanceType)
throws Exception {
_config = config;
_recordReader = dataSource.getRecordReader();
_dataSchema = config.getSchema();
_continueOnError = config.isContinueOnError();
+ _instanceType = instanceType;
Review Comment:
Maybe we could add a precondition check to verify that the instance type is
minion or server?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
