Centos7安裝 Hadoop（單節點）

1.Hadoop簡介

• Hadoop是一個由Apache基金會所開發的開源分布式系統基礎框架，使用Java開發，是處理大規模數據的軟件平台。
• Hadoop可以從單一節點擴展到上千節點。用戶可以在不了解分布式底層細節的情況下，開發分布式程序。充分利用集群的威力進行高速運算和存儲。
• 本次安裝的是單點問題

2.安裝Java JDK

• 由於Hadoop使用Java編寫，要運行它我們需要安裝Java運行環境
• jdk下載地址:         http://www.oracle.com/technetwork/java/javase/downloads/index.html
• hadoop下載地址: https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/core/

#1.把事先准備好的包放到/usr/local/src下
[root@hadoop_master src]# ll
total 445732
-rw-r--r-- 1 root root 266688029 Nov 17 14:08 hadoop-2.7.4.tar.gz
-rw-r--r-- 1 root root 189736377 Nov 13 15:37 jdk-8u151-linux-x64.tar.gz
[root@hadoop_master src]# tar xf jdk-8u151-linux-x64.tar.gz 
[root@hadoop_master src]# mv jdk1.8.0_151/  /usr/local/jdk
#2.在環境變量追加
export JAVA_HOME=/usr/local/jdk
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/jre:$JRE_HOME
export PATH=$JAVA_HOME/bin:$PATH
#3.創建用戶
[root@hadoop_master src]# useradd hadoop
[root@hadoop_master src]# passwd hadoop
Changing password for user hadoop.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.

3.解壓hadoop包

[root@hadoop_master src]# tar xf hadoop-2.7.4.tar.gz 
[root@hadoop_master src]# mv hadoop-2.7.4 /usr/local/hadoop
[root@hadoop_master src]# chown -R hadoop:hadoop /usr/local/hadoop/

4.配置hadoop

[root@hadoop_master src]# su - hadoop
[hadoop@hadoop_master ~]$ vi .bashrc 
export JAVA_HOME=/usr/local/jdk
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
[hadoop@hadoop_master ~]$ source .bashrc 
[hadoop@hadoop_master ~]$  cd $HADOOP_HOME/etc/hadoop
[hadoop@hadoop_master hadoop]$ vi hadoop-env.sh
export JAVA_HOME=/usr/local/jdk
[hadoop@hadoop_master hadoop]$ grep "JAVA" hadoop-env.sh
# The only required environment variable is JAVA_HOME.  All others are
# set JAVA_HOME in this file, so that it is correctly defined on
export JAVA_HOME=/usr/local/jdk    #修改后的結果
#HADOOP_JAVA_PLATFORM_OPTS="-XX:-UsePerfData $HADOOP_JAVA_PLATFORM_OPTS"

• 編輯core-site.xml文件

<configuration>
<property>
  <name>fs.default.name</name>
    <value>hdfs://localhost:9000</value>
</property>
</configuration>

• 編輯hdfs-site.xml文件：

<configuration>
<property>
 <name>dfs.replication</name>
 <value>1</value>
</property>
 
<property>
  <name>dfs.name.dir</name>
  <value>file:///usr/local/hadoop/hadoopdata/namenode</value>
</property>
 
<property>
  <name>dfs.data.dir</name>
  <value>file:///usr/local/hadoop/hadoopdata/datanode</value>
</property>
 
</configuration>

• 創建目錄

[hadoop@hadoop_master hadoop]$ mkdir -p /usr/local/hadoop/hadoopdata/namenode  /usr/local/hadoop/hadoopdata/datanode /usr/local/hadoop/hadoopdata

• 創建mapred-site.xml文件：

[hadoop@hadoop_master hadoop]$ cp mapred-site.xml.template mapred-site.xml
<configuration>
 <property>
  <name>mapreduce.framework.name</name>
  <value>yarn</value>
 </property>
</configuration>

• 編輯yarn-site.xml文件:配置了Hadoop單節點

[hadoop@hadoop_master hadoop]$ cat yarn-site.xml
<configuration>

<!-- Site specific YARN configuration properties -->
<property>
  <name>yarn.nodemanager.aux-services</name>
  <value>mapreduce_shuffle</value>
</property>
</configuration>

• 初始化HDFS文件系統：

[hadoop@hadoop_master hadoop]$ hdfs namenode -format

• Hadoop相關腳本:

[hadoop@hadoop_master hadoop]$ cd ../../sbin/
[hadoop@hadoop_master sbin]$ ls
distribute-exclude.sh  hdfs-config.sh           refresh-namenodes.sh  start-balancer.sh    start-yarn.cmd  stop-balancer.sh    stop-yarn.cmd
hadoop-daemon.sh       httpfs.sh                slaves.sh             start-dfs.cmd        start-yarn.sh   stop-dfs.cmd        stop-yarn.sh
hadoop-daemons.sh      kms.sh                   start-all.cmd         start-dfs.sh         stop-all.cmd    stop-dfs.sh         yarn-daemon.sh
hdfs-config.cmd        mr-jobhistory-daemon.sh  start-all.sh          start-secure-dns.sh  stop-all.sh     stop-secure-dns.sh  yarn-daemons.sh

• 啟動Hadoop服務

[hadoop@hadoop_master sbin]$ start-dfs.sh     #啟動Hadoop
[hadoop@hadoop_master sbin]$ start-yarn.sh   #啟動yarn
[hadoop@hadoop_master sbin]$ jps                  #查看服務狀態
3664 NameNode
4368 NodeManager
4771 Jps
3785 DataNode
3945 SecondaryNameNode
4090 ResourceManager

• 登錄驗證:
• http://10.0.0.60:50070  #namenode端口
• http://10.0.0.60:8088    #所有項目訪問