Jackie-Jiang opened a new issue, #10069: URL: https://github.com/apache/pinot/issues/10069
Force commit is executed asynchronously leveraging the Helix messages. Currently there is no way to tell whether force commit is finished or not. We want to add a controller API to indicate whether all the consuming segments are successfully committed. There are 2 solutions I can think of: 1. Similar to tracking the reload status, we store the force commit request on the controller in a map, and use an uuid to look up the submitted force commit request. We can store the consuming segments when the force commit request is submitted, and then check if all of them turn into ONLINE state in the ideal state. 2. Do not track anything, and return the smallest of all the last committed segments' creation time. User can use this timestamp to tell whether all the segments are recently committed. For streams with some partition can be ended (e.g. Kinesis), it might need special handling so that the ended partition is excluded. cc @jadami10 @siddharthteotia @mcvsubbu @sajjad-moradi for some discussion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
