Yarn Fair Scheduler配置

• 1、先關閉yarn, stop-yarn.sh
• 2、開啟fair機制：在yarn-site.xml中配置：

<property>
    <name>yarn.resourcemanager.scheduler.class</name>
    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>

• 3、在$HADOOP_HOME/etc/hadoop下創建文件fair-scheduler.xml,並且配置：

<?xml version="1.0"?>
<allocations>
<defaultQueueSchedulingPolicy>fair</defaultQueueSchedulingPolicy>
        <queue name="prod">
                <weight>40</weight>
                <schedulingPolicy>fifo</schedulingPolicy>
        </queue>
        <queue name="dev">
                <weight>60</weight>
                <queue name="eng"/>
                <queue name="science"/>
        </queue>
</allocations>

• 4、同步：
  scp fair-scheduler.xml yarn-site.xml hadoop-twq@slave1:~/bigdata/hadoop-2.7.5/etc/hadoop/
scp fair-scheduler.xml yarn-site.xml hadoop-twq@slave2:~/bigdata/hadoop-2.7.5/etc/hadoop/
• 5、啟動yarn, start-yarn.sh

• 6、通過WebUI查看配置是否成功
  

• 在DistributedCount.java中MapReduce程序指定隊列運行
  job.getConfiguration().set("mapreduce.job.queuename", "eng");
  運行中通過WebUI可以查看運行過程：