priyen-stripe commented on issue #17160: URL: https://github.com/apache/pinot/issues/17160#issuecomment-3502367938
@noob-se7en , I don't think so. first `firstStreamIngestionTimeMs`, by its name, implies its the timestamp of the first ingestion when data travels across multiple topics. also, you cannot do this in the plugin because we only care about the timestamp for data that pinot successfully ingests. our low-level consumer only streams in data and passes it to pinot via the messageBatch interface; pinot may filter it out or do any amount of things with it after wards and we want the lag tracking to be from that point, not from when our client consumes the data from the stream. it's the same reason ingestion lag is tracked for data that is consumed only (indexed on the segment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
