rhodo commented on PR #15345: URL: https://github.com/apache/pinot/pull/15345#issuecomment-2749267004
> One concern is that constantly reading the segment list of a large table itself may put pressure on the controller and zookeeper. Is it possible to track this metric when segments are added or removed ? A daily periodic job can refresh the absolute value if required. Another suggestion is to add a config param as getting the list is a heavy operation. From reading the code, it looks like ZooKeeper access via the property store’s getChildrenNames() method is already cached ([ref](https://sourcegraph.com/github.com/apache/helix/-/blob/helix-core/src/main/java/org/apache/helix/manager/zk/ZkCacheBaseDataAccessor.java?L682)). Also, segment status checkers run every 5 minutes by default. Let me know if this addresses your concern. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
