jeffreyliu34 commented on a change in pull request #8273:
URL: https://github.com/apache/pinot/pull/8273#discussion_r819895640
##########
File path:
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java
##########
@@ -279,6 +294,23 @@ protected void flattenMap(GenericRow record, List<String>
columns) {
}
}
+ /**
+ *
+ */
+ @VisibleForTesting
+ protected void dropPrefixes(GenericRow record) {
+ List<String> columns = new
ArrayList<>(record.getFieldToValueMap().keySet());
Review comment:
Good point. Since this was mainly intended for unnested fields, I was
thinking it to be a rare edge case for prefixes matching regular columns since
they'd lack the delimiter (ie "after." vs "after"), and thought it might help
to have more flexibility to drop prefixes from regular columns if desired. I
understand the concern though, so it can be updated it to drop prefixes only
from columns generated by the ComplexTypeTransformer.
Initially, I thought about doing this when generating the `flattenName`, but
this would affect the `unnestCollection` logic here -
https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java#L154-L157
since we're flattening the map first (where we would have dropped prefixes)
before unnesting any collections.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
