基於HADOOP3.0+Centos7.0的yarn基礎配置；
執行步驟：（1）配置集群yarn （2）啟動、測試集群（3）在yarn上執行wordcount案例
一、配置yarn集群
1.配置yarn-env.sh
添加：export JAVA_HOME=/opt/module/jdk1.8.0_144
2.配置yarn-site.xml
1 <!-- reducer獲取數據的方式--> 2 <property> 3 <name>yarn.nodemanager.aux-services</name> 4 <value>mapreduce_shuffle</value> 5 </property> 6 7 <!-- 指定YARN的ResourceManager的地址--> 8 <property> 9 <name>yarn.resourcemanager.hostname</name> 10 <value>hadoop101</value> 11 </property> 12 13 <!--在etc/hadoop/yarn-site.xml文件中，修改檢查虛擬內存的屬性為false--> 14 <property> 15 <name>yarn.nodemanager.vmem-check-enabled</name> 16 <value>false</value> 17 </property> 18 19 <!--引入hadoop路徑--> 20 <property> 21 <name>yarn.application.classpath</name> 22 23 <value> 24 /opt/hadoop-2.6.0/etc/hadoop, 25 /opt/hadoop-2.6.0/share/hadoop/common/*, 26 /opt/hadoop-2.6.0/share/hadoop/common/lib/*, 27 /opt/hadoop-2.6.0/share/hadoop/hdfs/*, 28 /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/*, 29 /opt/hadoop-2.6.0/share/hadoop/mapreduce/*, 30 /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/*, 31 /opt/hadoop-2.6.0/share/hadoop/yarn/*, 32 /opt/hadoop-2.6.0/share/hadoop/yarn/lib/* 33 </value> 34 </property>
3.配置：mapred-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_144
4.配置mapred-site.xml
<!-- 指定mr運行在yarn上--> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>yarn.application.classpath</name> <value> /opt/hadoop-2.6.0/etc/hadoop, /opt/hadoop-2.6.0/share/hadoop/common/*, /opt/hadoop-2.6.0/share/hadoop/common/lib/*, /opt/hadoop-2.6.0/share/hadoop/hdfs/*, /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/*, /opt/hadoop-2.6.0/share/hadoop/mapreduce/*, /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/*, /opt/hadoop-2.6.0/share/hadoop/yarn/*, /opt/hadoop-2.6.0/share/hadoop/yarn/lib/* </value> </property>
二、啟動集群
1.啟動hdfs集群（namenode+datanode）
2.啟動yarn集群（resourceManger+nodeManager）
sbin/start-yarn.sh or pasting
yarn --daemon start resourcemanager
yarn --daemon start nodemanager
3.jps+回車 查看進程
訪問 http://192.168.1.101:8088/cluster
三、運行案例
1.刪除文件系統上原來的output文件
hdfs dfs -rm -R /user/atguigu/output
2.執行mapreduce程序
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar wordcount /user/atguigu/input /user/atguigu/output
3.查看運行結果
hdfs dfs -cat /user/atguigu/output/*
歷史服務器配置
配置mapred-site.xml
<property> <name>mapreduce.jobhistory.address</name> <value>0.0.0.0:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>0.0.0.0:19888</value> </property>
啟動命令：
mapred --daemon stop historyserver
or
sbin/mr-jobhistory-daemon.sh start historyserver
訪問地址： http://ip:19888/jobhistory