jadami10 opened a new pull request, #11534: URL: https://github.com/apache/pinot/pull/11534
This is a fix for #11533. First, we shutdown the ingestion delay first when the `RealtimeTableDataManger` is shutdown. This way the gauges are removed and stop reporting lag instead of waiting for all segments to get shutdown. Second, we stop emitting ingestion lag in `updateIngestionDelay` once the `RealtimeTableDataManger` is shutdown. This way, segments that have stopped consuming cannot keep updating ingestion delay. I've tested this on a QA cluster consuming thousands of events per second. Before this, shutting down a server would show ~10 minutes of ingestion lag. After this, the ingestion lag stays stable and eventually stops reporting once all segments are destroyed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
