rohityadav1993 opened a new pull request, #17287: URL: https://github.com/apache/pinot/pull/17287
`Configuration` `release-notes` Relates to #14686 ### Problem In infrastructures where the underlying host can be shared by multiple technologies. Deepstore downloads in predownload step with high parallelism can lead to IO bandwidth exhaustion. Alternatively, hosts with higher bandwidth can allow a higher parallelism compared to the hard-coded value (numProcessors * 3). This PR adds the capability to set parallelism using a server-level config. ### Changes: - Define parallelism when pre-downloading segments from deepstore as `pinot.server.predownload.parallelism = 10` - If any invalid config (<=0) is defined or missing then fallback to existing logic of using `parallellism = numProcessors * 3` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
