glkz opened a new pull request, #198: URL: https://github.com/apache/iceberg-go/pull/198
This PR introduces an option to set `MaxConcurrency` for the `Scanner`, allowing users to control the level of concurrent downloads in Scanner. This configuration can be beneficial for workloads running multiple simultaneous queries. Additionally, this PR changes the default value of max concurrency from `runtime.NumCPU()` to `runtime.GOMAXPROCS`. While both are initially set to the number of available CPUs, `GOMAXPROCS` is adjustable by users, providing greater flexibility. This change addresses a limitation of using `runtime.NumCPU()`, which does not account for cgroup limits. For instance, in Kubernetes environments, using `runtime.NumCPU()` may incorrectly assume access to all CPU cores on a node, rather than only the cores allocated to the specific pod. This mismatch can lead to performance degradation in Kubernetes-deployed applications. See [uber-go/automaxprocs](https://pkg.go.dev/go.uber.org/automaxprocs). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
