somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2445856594
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
MultipleTreesBuilder builder = new
MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
buildMode)) {
builder.build();
+ } catch (Exception e) {
+ String tableNameWithType = _config.getTableConfig().getTableName();
+ LOGGER.error("Failed to build star-tree index for table: {},
skipping", tableNameWithType, e);
+ MinionMetrics.get().addMeteredTableValue(tableNameWithType,
MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
Review Comment:
i actually do see the `build()` method called by the realtime segment
conversion path (`RealtimeSegmentConverter`):
```
/**
* Common method to build segment with the provided record reader
*/
private void buildSegmentWithReader(SegmentIndexCreationDriverImpl driver,
SegmentGeneratorConfig genConfig,
RecordReader recordReader, int[] sortedDocIds, boolean
useCompactedReader,
@Nullable ThreadSafeMutableRoaringBitmap validDocIdsSnapshot)
throws Exception {
RealtimeSegmentSegmentCreationDataSource dataSource;
if (useCompactedReader) {
// For compacted readers, use the constructor that takes sortedDocIds
and pass the validDocIds snapshot
dataSource = new
RealtimeSegmentSegmentCreationDataSource(_realtimeSegmentImpl, recordReader,
sortedDocIds,
validDocIdsSnapshot);
} else {
// For regular readers, use the original constructor
dataSource =
new RealtimeSegmentSegmentCreationDataSource(_realtimeSegmentImpl,
(PinotSegmentRecordReader) recordReader);
}
driver.init(genConfig, dataSource,
TransformPipeline.getPassThroughPipeline(_tableName)); // initializes reader
if (!_enableColumnMajor) {
driver.build();
} else {
//buildByColumn uses validDocIds to skip invalid record while indexing
each column. We pass the validDocIds
// only if we are using compacted reader.
driver.buildByColumn(_realtimeSegmentImpl, useCompactedReader ?
validDocIdsSnapshot : null);
}
}
```
so I do think we need to pass some context on whether this is called from
minion code path or server 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
