jadami10 commented on PR #9994: URL: https://github.com/apache/pinot/pull/9994#issuecomment-1399107604
> I guess first question is what is the stream involved? This should return zero when we are not observing events:messagesAndOffsets.getMessageCount(); We ingest from kafka, but we talk through a proxy that consumes events from multiple kafka clusters. I've confirmed our subclass of `MessageBatch` returns the size of the records list we get, so it should be returning 0. > Can you check also what is the fetchTimeout config for your stream? We set it to 500ms. Our stream ingestion plugin is fairly strange. The upstream consume API we use is async, so we request records and wait either 500ms or until we've received 1000 records before returning. is this feature using offsets at all? I know we've had some weird issues in the past where there's an off by 1 depending on what you consider the "latest" offset -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
