mkvenkatesh commented on issue #14951: URL: https://github.com/apache/iceberg/issues/14951#issuecomment-3939920958
> The only time I've ever seen BindExceptions was when setting up a socket to listen on a port and the port was in use. > > Here maybe there's something up with your local /etc/hosts setting or similar such that httpclient cannot bind to an ephemeral port on what it thinks is your IP address. > > I'd be very surprised if this was anything other than client-side config issues. Thank you. Moving to 1.10.1 didn't improve things so we fiddled with the HTTP client settings and `iceberg.worker.delete-num-threads` and the BindExceptions have disappeared. Final settings that helped for us -- http client configs ``` http-client.apache.max-connections: 50 http-client.apache.connection-time-to-live-ms: 300000 http-client.apache.connection-max-idle-time-ms: 30000 http-client.apache.tcp-keep-alive-enabled: true ``` -- spark conf ``` --conf spark.executor.extraJavaOptions="-Diceberg.worker.delete-num-threads=8" ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
