somandal opened a new pull request, #15001: URL: https://github.com/apache/pinot/pull/15001
Today the config: `pinot.server.instance.table.level.max.parallel.segment.downloads` is set up based on the server configuration. This PR aims to make this config dynamically configurable via the ZK cluster configs so that a server restart is not required to pick up changes to this configuration. Another change this PR makes is to always create the semaphore to control the parallelism and removes the behavior when this configuration is set to `-1` to indicate downloads can be unlimited. Instead it sets a high default value for parallel downloads instead. To handle backward compatibility, in case the server configuration sets this to `-1` the init() code will default it to a high default value instead. ZK cluster config updates must set it to be > 0 though. The above should be okay as the maximum threads that could be downloading segments at any given time will be based on the value of STATE_TRANSITIONS.maxThreads (which is 40 by default) + max.parallel.refresh.threads cc @Jackie-Jiang @klsince @npawar @jackjlli @jasperjiaguo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
