jadami10 commented on PR #9244: URL: https://github.com/apache/pinot/pull/9244#issuecomment-1220771563
Hey @navina, thank you for looking! I agree that this would likely be better as a table level check and maybe even happen continuously through the lifecycle of the server so the broker is aware what servers are fresh enough to server data (I think Kafka does something similar with its brokers). That said, I think Pinot currently lacks some of the protection needed to do something like that. I know at the scale of our events, even 1 table catching up utilizes a large amount of CPU for example. And while tables can have their consumption rate limited, it's up to the user to make sure that's tuned correctly rather than being able to rely on Pinot to ensure there's always enough CPU to serve queries quickly. This was also much easier to implement as a starting point. Kudos to whoever wrote the Offset based startup checker because I was able to follow that pattern to easily do this as well. I realize the description also lacked some motivation. But we've seen cases where the Offset based checker take 10-15 minutes to catchup to that offset at server start, the server starts serving queries, but then we are 10-15 minutes behind still and the server is slow to serve queries until it actually finishes catching up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
