tibrewalpratik17 opened a new issue, #10552: URL: https://github.com/apache/pinot/issues/10552
We have a partial upsert table. The ingestion of the partial upsert table stopped after the server restarts. This can potentially be caused due to #10198. Thread dump shows the thread stuck at this: ``` java.lang.Thread.State: TIMED_WAITING (sleeping) -- at java.lang.Thread.sleep(java.base@11.0.15/Native Method) at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.consumeLoop(LLRealtimeSegmentDataManager.java:400) at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:633) at java.lang.Thread.run(java.base@11.0.15/Thread.java:829) ``` After the server restarts, the consuming segment messages are replayed. But in our case there is a record which comes out of order after which the replaying gets stuck in the above `consumeLoop` method. This was the original log message during normal consumption. ``` current comparison value: 1680617590, record comparison value: 1680617589), skipping updating the record ``` After this, I see consuming messages logs during the entire day but just after the server restart, there is no record beyond the timestamp of this log showing reloading the segment failed. I see logs of `No events came in, extending time by 1 hours` every one hour for this table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
