[GitHub] [incubator-pinot] cbalci opened a new pull request #6383: Introduce 'LOOKUP' Transform Function

[GitBox]

cbalci opened a new pull request #6383:
URL: https://github.com/apache/incubator-pinot/pull/6383


   ## Description
   
   Introducing a new transform function; `LookupTransformFunction` as a part of 
Join project as described in [Lookup UDF Join In 
Pinot](https://docs.google.com/document/d/1InWmxbRqwcqIakzvoEWHLxtX4XR9H5L01256EbAUHV8/edit).
 This is a followup to the addition of `DimensionTableManager` in #6346. 
   
   **LOOKUP** is a regular transform function which uses the previously added 
DimensionTableDataManager to execute a lookup from a Dimension table. Call 
signature is as follows:
   
   ```
   LOOKUP(TableName, ColumnName, JoinKey, JoinValue [, JoinKey2, JoinValue2 
...])
   ```
   
   **TableName:** name of the dimension table which will be used
   **ColumnName:** column name from the dimension table to look up
   **JoinKey:** primary key column name for the dimension table. Note: Only 
primary key is supported for JoinKey
   **JoinValue:** primary key value
   *If the dimension table has more then one primary keys (composite PK), you 
can add more keys and values for the rest of the args: **JoinKey2**, 
**JoinValue2** ... etc.
   
   
    **Example Query:**
   ```
    select playerName, teamID, lookup('baseballTeams', 'teamName', 'teamID', 
teamID) from baseballStats limit 10
   ```
    Above example joins the dimension table 'baseballTeams' into regular table 
'baseballStats' on 'teamID' key. Lookup function returns the value of the 
column 'teamName'.
   
   To see the function in action you can also fire `JoinQuickstart` and test it 
as follows:
   <img width="2553" alt="Screen Shot 2020-12-23 at 8 04 46 PM" 
src="https://user-images.githubusercontent.com/212284/103111451-f38dcc00-4601-11eb-8dab-beada1346a5f.png";>
   
   ## Testing 
   
   - Unit tests are included to cover basic functionality and I also added a 
sample usage (table creation + query) in `JoinQuickstart`. 
   - End to end functionality is also tested in a previous POC pull request 
[here](https://github.com/cbalci/incubator-pinot/pull/1).
   
   ## Documentation
   
   - Comprehensive documentation explaining the usage of 'Dimension' tables is 
being worked on and will be added to the 
https://github.com/pinot-contrib/pinot-docs repository in a separate PR.
   - LookupTransformFuncion usage and a sample query is added as a JavaDoc .


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org