jugomezv commented on PR #9994: URL: https://github.com/apache/pinot/pull/9994#issuecomment-1399056857
> > I tried deploying this change to one of our clusters with #10101 or #10121. We currently track ingestion lag via our custom stream ingestion plugin. For the most part these match up. But I still see an issue where partitions that have infrequent events continue to report forever increasing lag. From the code comments it seems that shouldn't be the case, but I want to make sure before I dig into our plugin to see if something isn't implemented as expected. Can you check also what is the fetchTimeout config for your stream? Default seems to be 5 seconds so for an inactive system I would expect a sawtooth graph that peaks at fetch timeout and returns to zero periodically in intervals of fetch timeout, if fetch timeout has been set to larger value then we would expect that larger value to be the peak of the sawtooth graph -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
