jugomezv commented on PR #10418: URL: https://github.com/apache/pinot/pull/10418#issuecomment-1471328453
> I don't think this is correct. Imagine we got some real messages, followed by some filtered messages, followed by some real messages. We should not set the delay to 0 in case all messages are filtered because it doesn't indicate we have already caught up. Am I missing something here? @Jackie-Jiang I see your point of view, but the situation I am trying to address here is constant repeated invocations of the loop where we filtered all messages, the current outcome is not correct correct either: an event comes in and we record that delay, then event batches are all filtered: the current outcome is time continues to ramp up to hours and hours and we are actually consuming. This is confusing and not correct. Do you have a better suggestion? We have seen this in some tables that are not actively being used and I think someone else reported earlier. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
