npawar commented on issue #6302: URL: https://github.com/apache/pinot/issues/6302#issuecomment-892072573
There is an API (check Task section on swagger) to trigger the movement from real-time to offline instead of waiting for the periodic task (you still need minions). However the realtimeToOffline task is not designed to achieve exactly what you want. It will not pick up the segments for movement unless they're completed segments. If it seems consuming segments, it will wait for them to complete. https://youtu.be/V_KNUUrS6DA (minute 33 onwards) explains that behavior. We might need to change the job's behavior to support this. On Tue, Aug 3, 2021, 7:04 AM Valentin Touffet ***@***.***> wrote: > I would prefer using API instead of minions, since I would do it as soon > as my job finished processing data, I want to end this job only when the > data is ready (which means when the segment is in the OFFLINE table in this > solution). > > Not an issue if the API supports it since we can automate it in our job. > > This solution seems to work for our use case. But downloading and > uploading segments could be a long task, isn't it? There is no other way? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/pinot/issues/6302#issuecomment-892012040>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AEWIC3NWSBA6K6TAQS5ZSEDT3AOQLANCNFSM4UIMSSVQ> > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> > or Android > <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> > . > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
