itschrispeck commented on issue #15056: URL: https://github.com/apache/pinot/issues/15056#issuecomment-2803249791
I was recently exploring this change too, for a slightly different reason - we use `llcpartitionconsuming` metric to alert when ingestion is stopped for a long period of time. It seems that when CONSUMING state is handled early, it can trigger our alerts. The original change looks like it would solve our case. IIUC the main concerns are for data freshness. I wanted to discuss if we can still make the change and suggest users use segment status checker with offset/freshness configs to more deterministically handle startup behavior, or if we truly need multiple StateModels. Do you have any input here? cc: @Jackie-Jiang Some metrics illustrating the problem we're trying to solve. `llcpartitionconsuming` is 0 when no rows are being consumed in the first graph. <img width="945" alt="Image" src="https://github.com/user-attachments/assets/52c2de77-ad6b-4c24-a2c0-1d8964c1f076"; /> <img width="949" alt="Image" src="https://github.com/user-attachments/assets/504ce95e-106e-450b-9507-d5e91e163483"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
