[GitHub] [incubator-pinot] yupeng9 opened a new pull request #6845: Add the complex data type transformer for complex type handling

[GitBox]

yupeng9 opened a new pull request #6845:
URL: https://github.com/apache/incubator-pinot/pull/6845


   Part 1 for the complex type handling. Design doc: 
https://docs.google.com/document/d/1L6d0k_yF4TbFWHt5IJ5eYYjJZboRnJTBrQYRB2an9iI/edit#
   
   ## Description
   Add the transformer to transform the complex data type, which includes map 
flattening and collection unnesting.
   
   With the transformation, the complex struct `group` in the meetupRsvp 
example are flattened into primitive types, and can be queried like the 
following.
   
   
![image](https://user-images.githubusercontent.com/13425258/116165197-3515ad00-a6b0-11eb-9219-e4175e061121.png)
   
   **<code>release-notes</code>** 
   Added configs  `complexTypeHandlingConfig` to `ingestionConfig`.
   
   for example
   ```
              complexTypeHandlingConfig: {
                 "mode": "FLATTEN_MAP",
                 unnestConfig: ["group.group_topics"]
              }        
   
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org