siddharthteotia commented on a change in pull request #6494:
URL: https://github.com/apache/incubator-pinot/pull/6494#discussion_r566524925
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java
##########
@@ -414,4 +496,186 @@ protected void createColumnV1Indices(String column)
true/*hasDictionary*/, dictionaryElementSize,
true/*hasInvertedIndex*/, TextIndexType.NONE,
false/*hasFSTIndex*/, false/*hasJsonIndex*/);
}
+
+ /**
+ * Helper method to create the V1 indices (dictionary and forward index) for
a column with derived values.
+ * TODO:
+ * - Support chained derived column
+ * - Support raw derived column
+ * - Support multi-value derived column
+ */
+ private void createDerivedColumnV1Indices(String column, FunctionEvaluator
functionEvaluator,
+ List<ColumnMetadata> argumentsMetadata)
+ throws Exception {
+ // Initialize value readers for all arguments
+ int numArguments = argumentsMetadata.size();
+ List<ValueReader> valueReaders = new ArrayList<>(numArguments);
+ for (ColumnMetadata argumentMetadata : argumentsMetadata) {
+ valueReaders.add(new ValueReader(argumentMetadata));
+ }
+
+ try {
+ // Calculate the values for the derived column
+ Object[] inputValues = new Object[numArguments];
+ int numDocs = _segmentMetadata.getTotalDocs();
+ Object[] outputValues = new Object[numDocs];
+ for (int i = 0; i < numDocs; i++) {
+ for (int j = 0; j < numArguments; j++) {
+ inputValues[j] = valueReaders.get(j).getValue(i);
+ }
+ outputValues[i] = functionEvaluator.evaluate(inputValues);
+ }
+
+ FieldSpec fieldSpec = _schema.getFieldSpecFor(column);
+ StatsCollectorConfig statsCollectorConfig =
+ new StatsCollectorConfig(_indexLoadingConfig.getTableConfig(),
_schema, null);
+ ColumnIndexCreationInfo indexCreationInfo;
+ switch (fieldSpec.getDataType()) {
Review comment:
I suggest to separate the switch block into separate function if possible
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
