fallintoplace opened a new pull request, #1361: URL: https://github.com/apache/iceberg-go/pull/1361
## Summary Bound manifest merge bin processing with an errgroup concurrency limit instead of starting one goroutine per packed bin. The limit defaults to runtime.GOMAXPROCS(0) and can be tuned with the table property commit.manifest-merge.max-concurrency. Non-positive or invalid values fall back to the default. ## Why Large commits can produce many manifest merge bins. Launching all bin merges at once can spike goroutines, file handles, CPU, and memory during commit even though the work is naturally parallel only up to a reasonable worker limit. ## Testing - go test ./table -run 'TestManifestMerge(MaxConcurrency|GroupLimits)' -count=1 - go test ./table -count=1 - go test ./... -skip '^TestCreateAzureBucketManagedIdentityCredentialCalled$' - go run github.com/golangci/golangci-lint/v2/cmd/[email protected] run --timeout=10m ./table/... - git diff --check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
