somandal opened a new pull request, #16146: URL: https://github.com/apache/pinot/pull/16146
This PR builds on top of https://github.com/apache/pinot/pull/15008 to extend the resource utilization check to also check for too many primary keys and pause REALTIME ingestion when that is hit for any servers that table is assigned to. - A server side API is added to return the number of primary keys, which are calculated across all upsert and dedup enabled tables. Some API / interfaces changes for upsert / dedup has been made to support this - The primary key count threshold is disabled by default (set to -1) (even if the resource utilization checker is enabled) - Minion task creation is not paused when primary key count threshold is breached (whereas tasks cannot be created if the disk utilization threshold is crossed) - A bunch of unit tests have been added Testing done: Manual testing is still in progress. Will update this once testing is done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
