Jackie-Jiang commented on a change in pull request #6494:
URL: https://github.com/apache/incubator-pinot/pull/6494#discussion_r566535763
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java
##########
@@ -414,4 +496,186 @@ protected void createColumnV1Indices(String column)
true/*hasDictionary*/, dictionaryElementSize,
true/*hasInvertedIndex*/, TextIndexType.NONE,
false/*hasFSTIndex*/, false/*hasJsonIndex*/);
}
+
+ /**
+ * Helper method to create the V1 indices (dictionary and forward index) for
a column with derived values.
+ * TODO:
+ * - Support chained derived column
+ * - Support raw derived column
+ * - Support multi-value derived column
+ */
+ private void createDerivedColumnV1Indices(String column, FunctionEvaluator
functionEvaluator,
+ List<ColumnMetadata> argumentsMetadata)
+ throws Exception {
+ // Initialize value readers for all arguments
+ int numArguments = argumentsMetadata.size();
+ List<ValueReader> valueReaders = new ArrayList<>(numArguments);
+ for (ColumnMetadata argumentMetadata : argumentsMetadata) {
+ valueReaders.add(new ValueReader(argumentMetadata));
+ }
+
+ try {
+ // Calculate the values for the derived column
+ Object[] inputValues = new Object[numArguments];
+ int numDocs = _segmentMetadata.getTotalDocs();
+ Object[] outputValues = new Object[numDocs];
+ for (int i = 0; i < numDocs; i++) {
+ for (int j = 0; j < numArguments; j++) {
+ inputValues[j] = valueReaders.get(j).getValue(i);
+ }
+ outputValues[i] = functionEvaluator.evaluate(inputValues);
+ }
+
+ FieldSpec fieldSpec = _schema.getFieldSpecFor(column);
+ StatsCollectorConfig statsCollectorConfig =
+ new StatsCollectorConfig(_indexLoadingConfig.getTableConfig(),
_schema, null);
+ ColumnIndexCreationInfo indexCreationInfo;
+ switch (fieldSpec.getDataType()) {
Review comment:
I don't think separating them into separate functions is more readable
because it is not reused anywhere else, and keeping them together is more
concise and easier to read IMO.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
