jugomezv commented on PR #10418: URL: https://github.com/apache/pinot/pull/10418#issuecomment-1472666218
I slept over this and I think there are two possible paths we could take: 1.-Modify the stream API so that we can extract at least one time stamp of a batch of all filtered messages: this will enable the tracker to publish a precise metric. If this is too complex we can at least introduce 2: 2.-Emit a info log after a fixed number of contiguous all-filtered batches: although this is less desirable at least will enable us to check the logs and verify if a ever increasing ingestion delay metric is due to actual problem or contiguous batches of all-filtered messages. What do you guys think? @navina @Jackie-Jiang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
