J-HowHuang opened a new pull request, #15175: URL: https://github.com/apache/pinot/pull/15175
## Description Disk utilization check was added to controller's periodic task in https://github.com/apache/pinot/pull/15008. This change was to pause data ingestion automatically. Server can also suffer from the lack of disk space while rebalanced. Therefore, there should also be sufficient observability of disk utilization under the context of rebalancing. ## Current plans - [ ] Add a pre-check item to check if disk utilization exceeds the safe threshold for each server - [x] Add disk utilization for each server into rebalance result summary - [ ] Estimate or compute the actual size net change of the segments migration for each server - [ ] Expose disk utilization threshold in configuration - [ ] Add flag to enable disk utilization in rebalance API (also need to check if controller has `controller.enable.resource.utilization.check` configured true) ## Issues and discussions - Disk utilization change is not obvious if multiple servers mount on the same disk - Is it reasonable to take the disk utilization values that are fetched by the controller's periodic task? - If so, can we rely on the configurations (e.g. `controller.resource.utilization.checker.frequency`) or do we need to override some? - Alternative: make API requests to server instances while doing rebalance -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
