ssharman-cloud opened a new issue, #15540: URL: https://github.com/apache/pinot/issues/15540
In some strange scenarios, Pinot Server is keep on crashing with "CrashLoopBackOff" error, zookeeper is not able to communicate to the Pinot Server. Find below few observation from our side, but we could not come to a conclusion whether is this the cause of the error. 1. Number of Segments of the table was more than 30K (last time it was around 40K) 2. The disk (PVC) attached to the Pinot Servers were reached its size (2 * 8TB were fully occupied) While the logs indicates that the server is coming up and shutting down. -----logs----------- 2025/04/14 15:51:45.303 INFO [ServerInstance] [Start a Pinot [SERVER]] Starting data manager 2025/04/14 15:51:45.306 INFO [HelixInstanceDataManager] [Start a Pinot [SERVER]] Helix instance data manager started 2025/04/14 15:51:45.306 INFO [ServerInstance] [Start a Pinot [SERVER]] Finish starting data manager 2025/04/14 15:54:20.886 INFO [PinotServiceManager] [Thread-42] Shutting down Pinot Service Manager with all running Pinot instances... 2025/04/14 15:54:20.886 INFO [PinotServiceManager] [Thread-42] Shutting down Pinot Service Manager admin application... 2025/04/14 15:54:20.886 INFO [PinotServiceManager] [Thread-42] Deregistering service status handler ------logs--------- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
