jugomezv commented on PR #10418: URL: https://github.com/apache/pinot/pull/10418#issuecomment-1491242926
I tested this change in one of our perf test servers and observed no regression and using some tracing I observed that the special case we are trying to address is caught: 2023/03/30 22:35:02.106 INFO [LLRealtimeSegmentDataManager_ReviewThrottlerMessageStateChangeEvent__1__225__20230330T1242Z] [ReviewThrottlerMessageStateChangeEvent__1__225__20230330T1242Z] [pinot-server] [] =>updateIngestionDelay() indexedMessages = 0, noTransformedRows = true 2023/03/30 22:35:02.106 INFO [LLRealtimeSegmentDataManager_ReviewThrottlerMessageStateChangeEvent__1__225__20230330T1242Z] [ReviewThrottlerMessageStateChangeEvent__1__225__20230330T1242Z] [pinot-server] [] =>updateIngestionDelay(md) ingestion TS = 1680215702091, end to end TS = \ 1680215668086 In this case we indexed zero message and we had no transformed rows but we did have metadata for a null message and we recorded a timestamp. I also verified that we do record timestamps when actual messages are indexed (no regressions): 2023/03/30 22:40:37.336 INFO [LLRealtimeSegmentDataManager_CountableImpressionDiscounting__0__4664__20230329T2250Z] [CountableImpressionDiscounting__0__4664__20230329T2250Z] [pinot-server] [] =>updateIngestionDelay() indexedMessages = 9, noTransformedRows = false 2023/03/30 22:40:37.336 INFO [LLRealtimeSegmentDataManager_CountableImpressionDiscounting__0__4664__20230329T2250Z] [CountableImpressionDiscounting__0__4664__20230329T2250Z] [pinot-server] [] =>updateIngestionDelay(md) ingestion TS = 1680216037332, end to end TS = 1680215973760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
