Hi MJ, Extremely sorry for a late response...Had some infrastructure issues here...
I am using Hadoop 2.3.0. Actually when i was trying to solve this AppMaster issue, i came up with a strange observation. "STICKY SLOT" of app-master to only the data node at the Master node if i set the following parameters along with *yarn.resourcemanager.hostname* *yarn.resourcemanager.address to master:8034* *yarn.resourcemanager.scheduler.address to master:8030* *yarn.resourcemanager.resource-tracker.address to master:8025 *across all the slave nodes. The default values can be found here... https://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml I came up with this strange observation though...If i dont set this 3 values i had described above, always in the slave nodes if the appmaster is launced, it tries connecting to the resource manager to the default 0.0.0.0 and not to the specified one. But with this values set, app-master is always launced in the master and everything seems fine...So i brought the datanode in the master down and checked as to what is happening... Strange, the jobs are not even assigned to any appmaster...but i think appmaster doesnt have any property of getting sticky...resource manager looks for a free container and goes ahead launching it... So now these things needs to be resolved: 1) Why does it work fine if the 3 mentioned values are set in the slave nodes and app master gets launced only in the master node' 2) If no data node is there in the master, then application doesnt get assigned at all to any app master. I have attached my config files for your reference...[Renamed for better reading n understanding] Thanks for your response !! On Thu, Mar 6, 2014 at 7:34 AM, Mingjiang Shi <[email protected]> wrote: > Sorry, it should be accessing http://<node_manager_ip>:8042/conf to check > the value of yarn.resourcemanager. > scheduler.address on the node manager. > > > On Thu, Mar 6, 2014 at 9:36 AM, Mingjiang Shi <[email protected]> wrote: > >> Hi Sai, >> A few questions: >> 1. which version of hadoop are you using? yarn.resourcemanager.hostname >> is a new configuration which is not available old versions. >> 2. Does your yarn-site.xml contains >> yarn.resourcemanager.scheduler.address? If yes, what's the value? >> 3. or you could access http://<resource_mgr>:8088/conf to check the >> value of yarn.resourcemanager.scheduler.address. >> >> >> >> On Thu, Mar 6, 2014 at 3:29 AM, Sai Prasanna <[email protected]>wrote: >> >>> Hi, >>> >>> I have a five node cluster. One master and 4 slaves. Infact master also >>> has a data node running. When ever app master is launched in the master >>> node, simple wordcount program runs fine. But if it is launched in some >>> slave nodes, the progress of the application gets hung. >>> The problem is, though i have set the yarn.resourcemanager.hostname to >>> the ip-address of the master, the slave connects only to the default, >>> 0.0.0.0:8030. >>> What could be the reason ??? >>> >>> I get the following message in the logs of app.master in web-UI. >>> *"...Configuration: job.xml:an attempt to override final parameter: >>> mapreduce.job.end-notification.max.retry.interval; Ignoring.* >>> >>> >>> *2014-03-05 20:15:50,597 WARN [main] >>> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final >>> parameter: mapreduce.job.end-notification.max.attempts; Ignoring. >>> 2014-03-05 20:15:50,603 INFO [main] org.apache.hadoop.yarn.client.RMProxy: >>> Connecting to ResourceManager at /0.0.0.0:8030 >>> <http://0.0.0.0:8030>2014-03-05 20:15:56,632 INFO [main] >>> org.apache.hadoop.ipc.Client: Retrying connect to server: >>> 0.0.0.0/0.0.0.0:8030 <http://0.0.0.0/0.0.0.0:8030>. Already tried 0 >>> time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >>> sleepTime=1000 MILLISECONDS)"* >>> >>
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<property>
<name>mapreduce.job.reduces</name>
<value>3</value>
<description>~~~~~~~~~~~~~I changed it so that multiple reduce tasks can be launched~~~~~~~~~~~~</description>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>128</value>
<description>Minimum limit of memory to allocate to each container request at the Resource Manager.</description>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>2048</value>
<description>Maximum limit of memory to allocate to each container request at the Resource Manager.</description>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-vcores</name>
<value>1</value>
<description>The minimum allocation for every container request at the RM, in terms of virtual CPU cores. Requests lower than this won't take effect, and the specified value will get allocated the minimum.</description>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-vcores</name>
<value>2</value>
<description>The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value.</description>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>4096</value>
<description>Physical memory, in MB, to be made available to running containers</description>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>4</value>
<description>Number of CPU cores that can be allocated for containers.</description>
</property>
</configuration>
<!-- <property>
<name>yarn.resourcemanager.hostname</name>
<value>192.168.1.11</value>
<description>RM host name</description>
</property>
-->
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>mapreduce.job.reduces</name>
<value>5</value>
<description>~~~~~~~~~~~~~I changed it so that multiple reduce tasks can be launched~~~~~~~~~~~~</description>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>yarn.app.mapreduce.am.resource.mb</name>
<value>1024</value>
</property>
<property>
<name>yarn.app.mapreduce.am.command-opts</name>
<value>-Xmx768m</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
<description>Execution framework.</description>
</property>
<property>
<name>mapreduce.map.cpu.vcores</name>
<value>1</value>
<description>The number of virtual cores required for each map task.</description>
</property>
<property>
<name>mapreduce.reduce.cpu.vcores</name>
<value>1</value>
<description>The number of virtual cores required for each map task.</description>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>1024</value>
<description>Larger resource limit for maps.</description>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx768m</value>
<description>Heap-size for child jvms of maps.</description>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>1024</value>
<description>Larger resource limit for reduces.</description>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx768m</value>
<description>Heap-size for child jvms of reduces.</description>
</property>
<property>
<name>mapreduce.jobtracker.address</name>
<value>192.168.1.11:8021</value>
<!-- <value>jobtracker.alexjf.net:8021</value>!-->
</property>
<property>
<name>mapreduce.jobtracker.http.address</name>
<value>192.168.1.11:8021</value>
</property>
</configuration>
hosts-master
Description: Binary data
hosts-slaves
Description: Binary data
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>yarn.app.mapreduce.am.resource.mb</name>
<value>1024</value>
</property>
<property>
<name>yarn.app.mapreduce.am.command-opts</name>
<value>-Xmx768m</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
<description>Execution framework.</description>
</property>
<property>
<name>mapreduce.map.cpu.vcores</name>
<value>1</value>
<description>The number of virtual cores required for each map task.</description>
</property>
<property>
<name>mapreduce.reduce.cpu.vcores</name>
<value>1</value>
<description>The number of virtual cores required for each map task.</description>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>1024</value>
<description>Larger resource limit for maps.</description>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx768m</value>
<description>Heap-size for child jvms of maps.</description>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>1024</value>
<description>Larger resource limit for reduces.</description>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx768m</value>
<description>Heap-size for child jvms of reduces.</description>
</property>
<property>
<name>mapreduce.jobtracker.address</name>
<value>192.168.1.11:8021</value>
<!-- <value>jobtracker.alexjf.net:8021</value>!-->
</property>
</configuration>
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>550</value>
<description>Minimum limit of memory to allocate to each container request at the Resource Manager.</description>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>2048</value>
<description>Maximum limit of memory to allocate to each container request at the Resource Manager.</description>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-vcores</name>
<value>1</value>
<description>The minimum allocation for every container request at the RM, in terms of virtual CPU cores. Requests lower than this won't take effect, and the specified value will get allocated the minimum.</description>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-vcores</name>
<value>2</value>
<description>The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value.</description>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>2200</value>
<description>Physical memory, in MB, to be made available to running containers</description>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>4</value>
<description>Number of CPU cores that can be allocated for containers.</description>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
<description>shuffle service that needs to be set for Map Reduce to run </description>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>192.168.1.11</value>
<description>The hostname of RM </description>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8034</value>
<description>The hostname of RM </description>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
<description>The hostname of RM </description>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8025</value>
<description>The hostname of RM </description>
</property>
</configuration>
