lnbest0707-uber commented on code in PR #13890:
URL: https://github.com/apache/pinot/pull/13890#discussion_r1733445582
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/SchemaConformingTransformerV2.java:
##########
@@ -439,18 +439,29 @@ private ExtraFieldsContainer processField(SchemaTreeNode
parentNode, Deque<Strin
public void generateTextIndexLuceneDocument(Map.Entry<String, Object> kv,
List<String> indexDocuments,
Integer mergedTextIndexDocumentMaxLength) {
String key = kv.getKey();
- String val;
// To avoid redundant leading and tailing '"', only convert to JSON string
if the value is a list or an array
if (kv.getValue() instanceof Collection || kv.getValue() instanceof
Object[]) {
+ // Add the entire array or collection as one string to the Lucene doc.
try {
- val = JsonUtils.objectToString(kv.getValue());
+ addLuceneDoc(indexDocuments, mergedTextIndexDocumentMaxLength, key,
JsonUtils.objectToString(kv.getValue()));
+ // To enable array contains search, we also add each array element
with the key value pair to the Lucene doc.
+ if (kv.getValue() instanceof Collection) {
+ for (Object o : (Collection) kv.getValue()) {
Review Comment:
Here we only do first level extraction, if the case is key:[v1, [v21, v22],
v3], it would become v1:key,[v21,v22]:key,v3:key. We need to call it out if
expected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
