vvivekiyer commented on PR #11699: URL: https://github.com/apache/pinot/pull/11699#issuecomment-1739782509
@Jackie-Jiang We are not using the PriorityBasedSchedulers. Just curious - can you point me to the race condition? About this change - We have a use case in Linkedin where we want to query production tables using adhoc channels (tools, scripts, reports, etc). Note that these adhoc queries are not from the site-facing applications but still need to be executed on production tables with some reasonably degraded latency. So we've built an internal scheduler (short term) that caps the amount of CPU resources given to these adhoc workloads while still letting the regular production workloads perform as usual. This is just a short term solution. For the long term, I'm planning to leverage the above idea to build a long-term policy based workload management solution. I'm writing a PEP document for the overall effort and I'll share it with OSS soon (week or two). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
