Hadoop的安裝和配置
非說明皆在master節點操作！視頻鏈接：https://www.bilibili.com/video/BV1sr4y1y7C8/
1、創建安裝目錄
mkdir hadoop
2、解壓文件到hadoop的安裝目錄
tar -xzvf hadoop-2.7.3.tar.gz -C /hadoop
3、配置環境變量
vi /etc/profile
#hadoop envirment
export HADOOP_HOME=/hadoop/hadoop-2.7.3
export CLASSPATH=\$CLASSPATH:\$HADOOP_HOME/lib
export PATH=$PATH:\$HADOOP_HOME/bin
source /etc/profile
4、修改hadoop-env.sh
cd /hadoop/hadoop-2.7.3/etc/hadoop/
vi hadoop-env.sh
export JAVA_HOME=/usr/java/jdk1.8.0_171
export HADOOP_CONF_DIR=/hadoop/hadoop-2.7.3/etc/hadoop
5、修改core-site.xml
vi core-site.xml
<property>
<name>fs.default.name</name>
<value>hdfs://master:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/hadoop/hadoop-2.7.3/hdfs/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131072</value>
</property>
<property>
<name>fs.checkpoint.period</name>
<value>60</value>
</property>
<property>
<name>fs.checkpoint.size</name>
<value>67108864</value>
</property>
6、修改yarn-site.xml
vi yarn-site.xml
<property>
<name>yarn.resourcemanager.address</name>
<value>master:18040</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:18030</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:18088</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:18025</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:18141</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
7、修改hdfs-site.xml
vi hdfs-site.xml
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/hadoop/hadoop-2.7.3/hdfs/name</value>
<final>true</final>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/hadoop/hadoop-2.7.3/hdfs/data</value>
<final>true</final>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>master:9001</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
8、修改mapred-site.xml
cp mapred-site.xml.template mapred-site.xml
vi mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
9、修改slaves文件
vi slaves
slave1
slave2
10、新建master文件
vi master
master
11、復制到slave1和slave2
scp -r /hadoop root@slave1:/
scp -r /hadoop root@slave2:/
修改slave1和slave2配置並生效（同3、）
12、格式化hadoop
先格式化再啟動Hadoop，只格式化一次
hadoop namenode -format
13、啟動Hadoop
同步時間：ntpd服務
啟動zookeeper:所有機器都啟動
啟動Hadoop：只在master角色機器啟動
/hadoop/hadoop-2.7.3/sbin/start-all.sh
關閉：
/hadoop/hadoop-2.7.3/sbin/stop-all.sh
關閉zookeepr
關閉Linux系統