If it is useful to anyone, I saw that this problem has been fixed since 2.9.0
with https://issues.apache.org/jira/browse/MAPREDUCE-6338
Il martedì 25 gennaio 2022, 16:56:16 CET, Hamado Dene
<[email protected]> ha scritto:
Hi hadoop community,
I am noticing that when a container is run on the nodemanager, it always gets a
random port.
On our side we have the firewall active on the machines. So we try where
possible to specify a range of ports in the configuration and then unblock them
on the firewall.In the mapred configuration I have already made the following
config:<property> <name>yarn.app.mapreduce.am.job.client.port-range</name>
<value>50000-50050</value></property>
<property> <name>yarn.app.mapreduce.am.webapp.port-range</name>
<value>50051-50100</value></property>
But when the container is launched, analyzing the syslog log (/ data / hadoop /
logs / userlogs / container_e42_1643125102241_0001_01_000012 / syslog) I see
the exception:
2022-01-25 15:41:06,666 WARN [main] org.apache.hadoop.ipc.Client: Failed to
connect to server: rzv-db13-hd/10.216.5.15:37055: retries get failed due to
exceeded maximum allowed retries number: 10java.net.NoRouteToHostException: No
route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method) at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at
org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:685)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:788)
at org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:410)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1550) at
org.apache.hadoop.ipc.Client.call(Client.java:1381) at
org.apache.hadoop.ipc.Client.call(Client.java:1345) at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:249)
at com.sun.proxy.$Proxy8.getTask(Unknown Source) at
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:137)
This is because port 37055 is not unlocked on the machine.verifying on the
rzv-db13-hd machine, I see that the service listening on that port is:
/usr/java/current8/bin/java
-Djava.io.tmpdir=/data2/namenodeFsimage/yarnData/usercache/magnews/appcache/application_1643115383937_0011/container_e37_1643115383937_0011_01_000001/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir=/data/hadoop/logs/userlogs/application_1643115383937_0011/container_e37_1643115383937_0011_01_000001
-Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
-Dhadoop.root.logfile=syslog -Xmx1024m
org.apache.hadoop.mapreduce.v2.app.MRAppMaster
Therefore the org.apache.hadoop.mapreduce.v2.app.MRAppMaster service always
generates a random port at each execution.Is it possible to specify a range of
ports for this App?
Thanks,
