sajjad-moradi opened a new issue #7274: URL: https://github.com/apache/pinot/issues/7274
Realtime consumption status checker is used at startup time in Pinot Server to define a catch-up period in which no query execution happens and stream consumers try to catch up to the latest messages available in different streams. The current realtime consumption status checker defines a time threshold for startup consumption. When the wait time after Pinot Server starts up passes that time threshold, the status checker returns ServiceStatus.GOOD. There are some ways to improve the existing behavior: - Add consumption rate statistics to the StatsHistory files of consuming segments. Then in startup period, consume greedily until the consumption rate converges to the rate available in the StatsHistory file. - At startup time, get the latest stream offset for each consuming segments and consume until the latest ingested offset is greater than the latest observed stream offset. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
