configure hadoop-0.22 fairscheduler

[Jameson Li]

I want to test version hadoop-0.22.
But when configurate the fairescheduler, I have some troublesome. The
fairscheduler is not efficient.
And I have configured this items in the mapred-site.xml, and also I hava
copy the fairscheduler jar file to the $HADOOP_HOME/lib:

  <property>
    <name>mapreduce.jobtracker.taskScheduler</name>
    <value>org.apache.hadoop.mapred.FairScheduler</value>
  </property>

<property>
        <name>mapred.fairscheduler.allocation.file</name>
        <value>conf/pools.xml</value>
</property>
<property>
        <name>mapred.fairscheduler.preemption</name>
        <value>true</value>
</property>
<property>
        <name>mapred.fairscheduler.assignmultiple</name>
        <value>true</value>
</property>
<property>
        <name>mapred.fairscheduler.poolnameproperty</name>
        <value>mapred.queue.name</value>
        <description>job.set("mapred.queue.name",pool); // pool is set to
either 'high' or 'low' </description>
</property>
<property>
        <name>mapred.queue.names</name>
        <value>default,aaa,bbb</value>
</property>

And the pools.xml in $HADOOP_HOME/conf 's content:

<?xml version="1.0"?>
<allocations>
  <pool name="putindb">
   <minMaps>72</minMaps>
   <minReduces>16</minReduces>
   <maxRunningJobs>20</maxRunningJobs>
   <weight>3.0</weight>
   <minSharePreemptionTimeout>60</minSharePreemptionTimeout>
  </pool>

  <pool name="machinelearning">
   <minMaps>9</minMaps>
   <minReduces>2</minReduces>
   <maxRunningJobs>10</maxRunningJobs>
   <weight>2.0</weight>
   <minSharePreemptionTimeout>60</minSharePreemptionTimeout>
  </pool>

  <pool name="default">
   <minMaps>9</minMaps>
   <minReduces>2</minReduces>
   <maxRunningJobs>10</maxRunningJobs>
   <weight>1.0</weight>
   <minSharePreemptionTimeout>60</minSharePreemptionTimeout>
  </pool>

  <defaultMinSharePreemptionTimeout>60</defaultMinSharePreemptionTimeout>
  <fairSharePreemptionTimeout>60</fairSharePreemptionTimeout>
</allocations>

Can someone help me?


专注于Mysql,MSSQL,Oracle,Hadoop