Copilot commented on code in PR #16031:
URL: https://github.com/apache/pinot/pull/16031#discussion_r2133434654
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskExecutor.java:
##########
@@ -145,9 +145,13 @@ protected List<SegmentConversionResult>
convert(PinotTaskConfig pinotTaskConfig,
SegmentProcessorConfig segmentProcessorConfig =
segmentProcessorConfigBuilder.build();
List<File> outputSegmentDirs;
_eventObserver.notifyProgress(_pinotTaskConfig, "Generating segments");
- outputSegmentDirs = new SegmentProcessorFramework(segmentProcessorConfig,
workingDir,
+ SegmentProcessorFramework framework = new
SegmentProcessorFramework(segmentProcessorConfig, workingDir,
recordReaderFileConfigs, Collections.emptyList(), new
DefaultSegmentNumRowProvider(Integer.parseInt(
-
configs.get(MinionConstants.UpsertCompactMergeTask.MAX_NUM_RECORDS_PER_SEGMENT_KEY)))).process();
+
configs.get(MinionConstants.UpsertCompactMergeTask.MAX_NUM_RECORDS_PER_SEGMENT_KEY))));
+ outputSegmentDirs = framework.process();
+ _eventObserver.notifyProgress(_pinotTaskConfig,
+ "transformation stats - incomplete:" +
framework.getIncompleteRowsFound() + ", dropped:" + framework
+ .getSkippedRowsFound() + ", sanitized:" +
framework.getSanitizedRowsFound());
Review Comment:
The `notifyProgress` call with transformation stats is duplicated across
multiple executors; consider extracting a helper method to format and emit
these stats for consistent behavior and easier updates.
```suggestion
emitTransformationStats(framework);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
