Neuw84 commented on code in PR #11383: URL: https://github.com/apache/iceberg/pull/11383#discussion_r1824255200
########## docs/docs/aws.md: ########## @@ -669,6 +669,13 @@ Users can use catalog properties to override the defaults. For example, to confi --conf spark.sql.catalog.my_catalog.http-client.apache.max-connections=5 ``` +**Note that for workloads with exceptionally high throughput against tables that S3 where you will likely to increase Retries, you will also like to increase the number of connections for the HTTP client** + +```shell +spark.sql.catalog.my_catalog.http-client.apache.max-connections=200 Review Comment: It is a thing of AWS SDK and Spark ( not specifically to EMR). If you use Spark on your laptop writing to S3 and you are on this high throughput write scenario you will likely like to tune the parameter. I mean, we are speaking about a AWS docs in this page ( the parameter is quite specific to the S3 client of the AWS SDK). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
