npawar opened a new pull request #6124: URL: https://github.com/apache/incubator-pinot/pull/6124
## Description https://github.com/apache/incubator-pinot/issues/5753 Here's the final piece of the feature for Pinot managed offline flows. This is the TaskGenerator which will create tasks of `realtimeToOfflineSegmentsTask` type. Typical usecase: You have setup a realtime table. You want to make it a hybrid table. Using this feature, you don't have to write your offline flows. Just set ``` "tableName": "myTable_REALTIME", "task": { "taskTypeConfigsMap": { "realtimeToOfflineSegmentsTask": { } } } ``` in your table config. The minion tasks will push data to the offline table, 1 day at a time. The window size, buffer, segment processing configs are configurable via table task config. ## Release Notes New feature for Pinot managed offline flows - a minion task which will automatically take data from the realtime table and push it to the offline table. ## Documentation WIP ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
